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information  driven  and  risk  based  prevention  and  response.  Given  the  diversity  of  NTR, 
distribution  of  “one  size  fits  all”  products  ensures  that  the  intelligence  will  fit  no  one’s 
needs. 

This  thesis  researches  the  impact  of  intelligence  contextual  background  fusion 
(CBF)  through  the  use  of  hyperlink  technology  and  evaluates  the  likelihood  of  hyperlink 
acceptance  by  NTR.  By  utilizing  DHS  and  FBI  customer  satisfaction  survey  questions  in 
“quality”  and  “value”  factors  along  with  previously  validated  Technology  Acceptance 
Model  (TAM)  questions  in  “ease  of  use”  and  “usefulness”  factors,  this  research  finds  that 
CBF  significantly  improves  both  perceived  value  and  quality,  and  finds  that  NTR 
overwhelmingly  prefer  a  CBF  product.  NTR  broadly  accepted  hyperlink  technology  in 
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I.  INTRODUCTION 


A.  PROBLEM  STATEMENT 

Multiple  initiatives,  including  the  Homeland  Security  Information  Network 
(HSIN)  and  the  Joint  Regional  Information  Exchange  System  (JRIES),  as  well  as 
supporting  directives  such  as  National  Capability  Specific  Priority  3.2.1:  Strengthening 
Infonnation  Sharing  and  Collaboration  Capabilities,  have  ensured  that  non-traditional 
recipients  (NTR)  in  law  enforcement,  fire,  emergency  medical  services  (EMS), 
emergency  management,  public  health  and  the  private  sector  are  receiving  unclassified 
intelligence  products  from  multiple  sources  including  the  Department  of  Homeland 
Security  (DHS),  the  Federal  Bureau  of  Investigation  (FBI),  Fusion  Centers,  Joint 
Terrorism  Task  Forces  (JTTF),  and  Terrorism  Early  Warning  Groups  (TEWG). 
However,  simply  pushing  intelligence  products  to  non  traditional  recipients  (NTR)  is  not 
enough.  Recipients  must  possess  enough  contextual  background  to  effectively  utilize  the 
intelligence  products  in  order  for  this  strategy  to  be  successful.  Unclassified  intelligence 
distribution  must  be  useful,  easy  to  use,  deliberate  and  coordinated,  maximizing 
recipients’  ability  to  effectively  utilize  the  intelligence,  regardless  of  the  time  of  day,  day 
of  week,  or  location  of  the  recipient.  We  can  do  better  in  this  regard.  As  discussed  in  a 
Markle  Foundation  Working  Group  Report,  Networking  of  Federal  Government 
Agencies  with  State  and  Local  Government  and  Private  Sector  Entities,  “...adequate 
context  for  homeland  security  providers  to  effectively  utilize  infonnation  is  specific, 
tailored  for  each  local  entity,  rapidly  disseminated,  and  does  not  overburden  recipients 
with  vague  or  irrelevant  information.”1  The  Final  9/11  Commission  Report  noted  the 
importance  of  context  in  decision  making,  reporting  that  the  President  was  provided 
intelligence  “news  without...  much  context”  prior  to  September  11,  2001,  contributing  to 
a  failure  of  decision  makers  to  recognize  that  Bin  Laden  posed  a  “novel  danger.”2  A 

1  Zoe  Baird,  Creating  a  Trusted  Network  for  Homeland  Security  (New  York,  New  York:  The  Markle 
Foundation,  2003),  56, 

http  ://www. markle. org/downloadable_assets/nstf_report2_part_two.pdf#search=%22adequate%20contextu 
al%20background%20homeland%20security%22  (accessed  September  24,  2006). 

2  Thomas  Kean  et  al.  The  9/11  Commission  Report.  (Washington,  D.C.:  United  States  General 
Accounting  Office,  2004),  119, 

http://files.fmdlaw.com/news.fmdlaw.com/hdocs/docs/91  lfinalrpt/9 11  report.pdf.  (accessed  January  27, 
2006). 
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similar  lack  of  context  in  the  evaluation  of  Iraq  intelligence  assessments  contributed  to  a 
failure  to  understand  Saddam  Hussein’s  perception  of  the  outside  world.3  At  the  local 
level,  in  2005,  Police  Chief  Patrick  Miller  found  that  “There  is  profound  belief  amongst 
law  enforcement  leaders  in  California  that  the  essential  mechanisms  for  information 
sharing  are  poor  at  best  and  arriving  at  some  consensus  on  how  to  fix  that  problem  was 
far  more  important  than  examining  leadership  models  and  future  impacts.”4  The  2007 
Department  of  Homeland  Security  Appropriations  Act  reinforces  this  charge  for  the 
Department  of  Homeland  Security  (DHS),  documenting  their  responsibility  to 
“...develop  better  methods  for  the  sharing  of  intelligence  with  State  and  local  law 
enforcement  agencies...”  with  similar  concerns,  the  General  Accounting  Office  has 
identified  at  least  1 1  separate  information  sharing  authorities  and  initiatives  since  9/1 1.5>6 

These  findings  and  directives  make  it  clear  that  one  mission  of  the  federal 
government  is  to  effectively  involve  all  public  safety  providers  in  homeland  security 
through  improved  information  sharing.  Effective  involvement  of  NTR  would  in  part 
result  in  an  improved  disaster  response  system,  reduction  of  man  made  disasters,  and 
“fusion”  of  emergency  response,  preparedness,  recovery,  mitigation,  and  critical 
infrastructure  protection.7  Critical  decision  making  is  not  limited  to  Chiefs  of  Police, 
Fire  or  Public  Health,  but  includes  professionals  at  every  level  of  organizations.  In  fact, 
those  “on  the  street,”  maintain  the  most  public  contact,  and  as  such,  make  some  of  the 
most  critical  homeland  security  decisions.  While  the  media  and  public  may  focus  on 
police  chiefs,  command  officers,  detectives,  or  other  high  visibility  law  enforcement 

3  Christopher  Andrew,  “Intelligence  Analysis  Needs  to  Look  Backwards  before  Looking  Forward,” 
History’  and  Policy. org,  1,  http://www.historyandpolicy.org/archive/pol-paper-print-23.html  (accessed  May 
19,2006). 

4  Patrick  Miller,  How  Can  We  Improve  Information  Sharing  Among  Local  Law  Enforcement 
Agencies,  Master’s  Thesis,  Naval  Postgraduate  School,  Monterey,  California,  March  2005. 

5  Department  of  Homeland  Security’  Appropirations  Act,  2007,  H.R.  5441,  109,  2,  July  17,  2006, 
S7632,  https://www.chds.us/courses/file.php/112/moddata/fomm/742/13414/DHS_Approp_Bill.txt 
(accessed  August  22,  2006). 

6  Government  Accountability  Office,  “Information  Sharing  the  Federal  Government  Needs  to 
Establish  Policies  and  Processes  for  Sharing  Terrorism-Related  and  Sensitive  but  Unclassified 
Information,”  United  States  Government  Accountability  Office  Report  to  Congressional  Requestors  6,  no. 
385  (March  17,  2006):  9,  http://www.gao.gov/new.items/d06385.pdf  (accessed  May  25,  2006). 

7  DEPARTMENT  OF  HOMELAND  SECURITY  APPROPRIATIONS  ACT,  2007,  H.R.  5441,  Senate 
Debate  on  the  Department  of  Homeland  Security  Appropriations  Act  2004,  1 . 
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positions  in  homeland  security,  patrol  officers  have  consistently  been  shown  to  have  the 
greatest  impact  on  crime.  As  described  by  Captain  Dennis  Potter,  Operations 
Commander  for  the  Columbine  school  shootings,  “When  something  really  bad  happens, 
the  initial  response  strategies  are  developed  from  the  front  seat  of  a  patrol  car,  not  from  a 
command  officers  desk...”8  Captain  Potter’s  observation  reflects  an  understanding  that 
intelligence  is  critical  in  day  to  day  tactical  decision  making,  not  just  in  long  term 
strategic  planning.  Research  documents  what  police  executives  know  about  the  critical 
nature  of  street  level  operations,  finding  that  80%  of  crimes  are  solved  by  patrol 
operations,  not  detective  work.9  Whether  planning  long  term  strategic  or  short  term 
tactical  operations,  it  is  critical  to  ensure  that  street  level  decision  makers  have  the 
intelligence  they  need.  This  is  the  audience  of  unclassified  intelligence  operations,  and 
while  it  may  not  be  practical  to  provide  classified  intelligence  to  these  millions  of 
professionals,  it  is  imperative  that  they  are  provided  the  very  best  unclassified 
intelligence  possible. 

This  research  does  not  attempt  to  detennine  whether  providing  classified  or 
unclassified  intelligence  to  NTR  is  an  effective  strategy.  The  choice  to  share  unclassified 
intelligence  products  has  already  been  made;  this  research  seeks  to  identify  the  impact  of 
implementing  contextual  background  fusion  (CBF)  with  intelligence  already  provided  to 
NTR  on  the  perceived  value  and  quality  of  that  intelligence.  Bill  Nolte,  Director  of 
Education  and  Training,  Office  of  the  Director  of  National  Intelligence,  reflected  on  the 
need  to  ensure  intelligence  usefulness,  “We  are  not  in  the  secrets  business;  we  are  in  the 
information  business...  The  buggy  whip  manufacturer  who  survived  realized  he  was  in 
the  transportation  business.”10  This  research  goes  further,  contending  that  we  are  in 
neither  the  secrets,  nor  information  business,  but  the  decision-making  business.  If 
intelligence  products  do  not  improve  decision  making,  the  IC  might  as  well  produce 

8  Dennis  Potter,  The  First  Three  Minutes:  Law  Enforcement ’s  Role  in  Crisis  Management  (F airfax, 
Virginia:  Commission  on  Accreditation  for  Law  Enforcement  Agencies,  2006),  1, 

http://64.233.187. 104/search?q=cache:0Y6GlXiUjgkJ:  www.calea.org/newweb/newsletter/No82/first3minu 
tes.htm+dennis+potter+the+first+three+minutes&hl=en&gl=us&ct=clnk&cd=l .  (accessed  October  7, 

2006). 

9  Tom  O’Conner  and  Paula  Baker,  Police  in  Society  Syllabus.  (Raleigh,  North  Carolina:  Wesleyan 
College,  2005),  1,  http://faculty.ncwc.edu/toconnor/205/2051ect08.htm  (accessed  October  18,  2006). 

10  Robert  Nolte,  “Remarks  to  Center  for  Flomeland  Defense  and  Security  Cohorts  0503,  0504“ 
(lecture.  Naval  Postgraduate  School,  Monterey,  February  27,  2006). 
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buggy  whips.  This  research  addresses  the  need  for  contextual  background  by  decision 
makers,  as  well  as  the  needs  of  the  IC,  who  depend  upon  continued  funding  by  decision 
makers. 

Policy  demands  that  more  information  be  shared;  for  that  information  to  be 
effectively  utilized,  decision  makers  must  perceive  both  value  and  quality  in  that 
information.11  This  research  does  not  fully  address  defining  the  needs  of  non-traditional 
intelligence  recipients,  another  critical  opportunity  for  research,  but  narrowly  focuses  on 
the  impact  of  CBF  of  open  source  information  sources  onto  unclassified  intelligence 
through  hyperlink  technology.  The  need  for  contextual  background  is  evident  at  every 
level  of  decision  making;  in  confirmation  hearings,  National  Director  of  Intelligence 
nominee  General  Hayden  reflected  on  lack  of  context  in  decision  making  in  response  to 
questioning  by  Senator  Feinstein:  “One  key  one  that  I  wanted  to  mention  when  the 
chairman  was  talking  about  it,  the  Iraq  WMD  estimate  was  essentially  worked  in  a  WMD 
channel.  It  was  absent  a  regional  or  cultural  context.  We  are  not  doing  that 
now. .  .We’re  not  doing  that  on  Iran.”12  (Emphasis  added) 

Understandably,  in  the  midst  of  terrorist  attack,  when  contextual  background  is 
most  needed  by  decision  makers,  those  with  personal  knowledge  of  relevant  context  and 
background  are  least  available  to  the  homeland  security  professionals  in  the 
approximately  18,000  law  enforcement  jurisdictions  who  may  seek  the  information,  let 
alone  other  homeland  security  disciplines;  professionals  who  will  use  whatever  context 
and  background  is  available  to  them  for  their  decision  making.  As  it  was  on  9/1 1,  police 
chiefs,  fire  chiefs,  private  security,  emergency  medical  services  and  public  health 
leadership  should  not  expect  return  phone  calls  from  any  federal  or  state  agency  that 

1 1  Maryam  Alavi,  “An  Assessment  of  the  Concept  of  Decision  Support  Systems  as  Viewed  by  Senior- 
Level  Executives,”  MIS  Quarterly  6,  no.  4  (December  1982):  1-9 

http://www.jstor.org.libproxy.nps.navy.mi1/view/02767783/ap010024/01a00040/77searchUrHhttp%3a//ww 
wjstor.org/search/BasicResults%3fhp%3d25%26si%3dl%26Query%3dperceived%2bvalue%2bin%2bdeci 
sion%2bmaking&frame=noframe&currentResult=02767783%2bap010024%2b01a00040%2b0%2cFF03& 
userID=837833bl@nps.navy.mil/01cc99333cb9210c8aec98ea&dpU3&config=jstor.  (accessed  July  21, 
2006). 

12  Senate  Select  Committee  on  Intelligence,  Hearing  of  the  Seriate  Select  Committee  on  Intelligence 
on  the  Nomination  of  General  Michael  Hayden  to  be  the  Director  of  Central  Intelligence,  109-112,  May 
16,  2006,  75-76,  http://www.fas.org/irp/congress/2006_hr/051806transcript.pdf  (accessed  September  5, 
2006). 
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might  be  able  to  provide  contextual  background,  as  these  agencies  will  be  inundated  by 
requests  and  operational  support  requirements.  When  information  is  most  needed, 
individuals  with  the  contextual  background  will  be  least  available,  busy  instead 
supporting  their  home  agencies  in  crisis  management.  Homeland  Security  professionals 
looking  for  contextual  background  are  left  to  their  own  resources  to  obtain  the 
information  they  need,  potentially  from  questionable  Internet  sources. 

In  one  effort  to  emphasize  the  importance  of  context,  the  U.S.  Government 
Printing  Office/CQ  Congressional  Reporting  Service  citations  include  the  phrase 
“Providing  government  documents  on  demand,  in  context”  explicitly  acknowledging  the 
critical  relationship  between  infonnation  sharing  and  context.  Intelligence  producers 
must  also  provide  intelligence  “on  demand,  in  context,”  unequivocally  affirming  the 
relationship  between  effective  information  sharing  and  context. 

Providing  unclassified  intelligence  “on  demand,  in  context”  is  critical  for  millions 
of  employees  in  NTR  disciplines  who  incorporate  that  context  into  day  to  day  decision 
making  in  public  contacts,  policy  development,  strategy  and  tactics.  In  the  absence  of 
CBF  by  intelligence  producers,  open  Internet  searches  of  unvetted  sites  by  intelligence 
recipients  can  prove  not  only  unreliable,  but  entirely  inaccurate.  For  instance,  a  Google 
search  that  might  be  completed  by  a  homeland  security  professional  searching  for 
contextual  background  on  aircraft  use  in  Islamic  terrorist  attacks,  “Islamic  terrorists 
kamikazes  weapon  aircraft”  leads  to  a  “Non  Aligned  Press  Network”  story  where  it  is 
reported  that  the  planes  on  9/11  were  flown  utilizing  remote  controls  by  individuals  in 
American  government.  False  Internet  postings  such  as  this  are  common  enough  that  the 
U.S.  State  Department  attempts  to  identify  such  misinformation  on 
USINFO.STATE.GOV.13  This  webpage  highlights  the  danger  of  utilizing  unvetted,  open 
searches  for  contextual  background;  If  the  intelligence  producer  fails  to  provide  CBF 
with  their  product,  NTR  may  find  very  authentic  looking  information  on  the  Internet  that, 
when  combined  with  timely,  accurate  and  actionable  intelligence,  produces  poor 
decisions. 

13  United  States  State  Department,  Did  a  Plane  Hit  the  Pentagon?  (Washington,  D.C.:  Department  of 
State,  2005),  1,  http://usinfo.state.gov/media/Archive/2005/Jun/28-581634.html  (accessed  September  15, 
2006). 
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The  context  and  background  utilized  by  decision  makers  must  be  reliable,  vetted 
information  that  is  perceived  as  high  quality  and  valuable,  and  it  must  be  consistent  with 
information  being  utilized  by  the  Department  of  Homeland  Security  (DHS),  the  Federal 
Bureau  of  Investigation  (FBI),  the  military,  and  other  state,  local  and  federal  homeland 
security  professionals.  Through  CBF,  the  IC  can  ensure  that  contextual  background  used 
NTR  is  reliable,  vetted,  and  consistent  with  the  henneneutic  of  the  producing  agency. 
The  IC  must  know  if  incorporating  a  CBF  system  with  intelligence  products  provided  to 
NTR  will  also  improve  the  perceived  value  and  quality  of  that  intelligence. 


B.  SPECIFIC  RESEARCH  OBJECTIVE 

The  overall  objective  of  this  research  is  to  improve  the  United  State’s  ability  to 
identify  and  mitigate  terrorist  threats  by  examining  the  impact  of  CBF  on  the  perceived 
value  and  quality  of  intelligence  products.  DHS  and  FBI  provide  unclassified 
intelligence  to  NTR  and  routinely  request  feedback  on  perceived  value  and  quality  as  part 
of  the  intelligence  cycle  focused  on  continual  counter  terrorism  improvement.  If  CBF  is 
shown  to  improve  DHS/FBI  defined  value  and  quality  factors,  the  technology  utilized 
must  be  evaluated  for  technology  acceptance  by  NTR.  Technology  Acceptance  Model 
research  has  shown  that  technology  must  be  perceived  as  both  easy  to  use  and  useful  if  it 
is  to  be  accepted  for  widespread  use.  Tested,  reliable,  and  generally  accepted  technology 
such  as  hyperlinks  allow  the  fusion  of  related  documents  in  a  manner  that  supports  a  user 
defined  experience  within  linking  constraints  established  by  the  author.  Context  and 
background  to  intelligence  products  must  meet  recipient  defined  needs.  As  noted  by  the 
Major  Cities  Police  Chiefs  Association  in  a  2002  report, 

...our  first  line  of  defense  against  terrorism— the  seven  hundred  thousand 
officers  on  the  street — (must)  be  given  adequate  training  and  background 
information  on  terrorism... our  local  and  state  officers  should  have 
background  knowledge... officers  should  have  the  ability  to  access 
national  data  banks. . . 14 


14  Edward  Tully,  Terrorism  the  Impact  on  State  and  Local  Law  Enforcement  (Alexandria,  VA: 
International  Association  of  Chiefs  of  Police,  2002),  5, 

http://www.neiassociates.org/mccintelligencereport.pdf  (accessed  September  2,  2006). 
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This  research  is  designed  to  detennine  the  impact  of  fusing  the  background 
information  demanded  by  the  Major  City  Police  Chiefs  on  the  perceived  value  and 
quality  of  intelligence  products  for  not  just  law  enforcement,  but  for  the  millions  of 
homeland  security  professionals  in  fire,  EMS,  public  health,  emergency  management,  and 
the  private  sector.  If  CBF  results  in  improved  value  and  quality,  technology  acceptance 
model  (TAM)  research  provides  a  framework  for  the  evaluation  of  the  potential  usage  of 
such  a  system;  likelihood  of  use  in  this  model  is  influenced  by  “perceived  ease  of  use,” 
and  “perceived  usefulness.”15  This  research  applies  previously  validated  TAM  processes 
to  evaluate  the  likelihood  that  NTR  will  utilize  hyperlink  technology  in  CBF  applications. 


C.  SIGNIFICANCE  OF  RESEARCH 

Allied  non-state  actors  in  the  private  sector  control  approximately  85%  of  the 
nation’s  critical  infrastructure  in  the  over  87,000  different  U.S.  jurisdictions.16  For  every 
federal  law  enforcement  officer,  there  are  approximately  five  public  sector  health 
professionals,  seven  state/local  law  enforcement  officers,  ten  firefighters  and  twenty-one 
private  security  professionals,  along  with  countless  other  public  works,  emergency 

17  18  19 

management  and  emergency  medical  professionals.  ’  ’  As  of  June  2000,  there  were 
approximately  708,022  state  and  local  law  enforcement  officers  in  the  United  States.20 
As  of  June  2002,  there  were  only  93,000  Federal  law  enforcement  officers,  or  less  than 
12%  of  sworn  law  enforcement.21  The  FBI,  lead  investigative  agency  for  domestic 
terrorism,  had  only  12,416  agents  as  of  October  2005,  or  approximately  1.5%  of  total  law 

15  F.  D.  Davis,  R.  Bagozzi  and  P.  R.  Warshaw,  “User  Acceptance  of  Computer  Technology,” 
Management  Science  35,  no.  8  (1989),  985,  http://links.jstor.org/sici7sicU0025- 
1909(198908)35%3A8%3C982%3AUAOCTA%3E2.0.CO%3B2-1  (accessed  May  26,  2006). 

16  George  Bush,  The  National  Security  Strategy  of  the  United  States  of  America  (Washington,  D.C.: 
Government  Printing  Office,  2002),  vii. 

17  State  and  Local  Law  Enforcement  Statistics,  (Washington,  D.C.:  Bureau  of  Justice  Statistics,  2006). 
1,  http://www.ojp. usdoj.gov/bjs/sandlle.htm  (accessed  December  1,  2005). 

18  Federal  Law  Enforcement  Statistics ,  (Washington,  D.C.:  Bureau  of  Justice  Statistics,  2006).  1, 
http://www.ojp.usdoj.gov/bjs/fedle.htm  (accessed  December  2,  2005). 

111  Association  of  State  and  Territorial  Health  Professionals,  State  Public  Health  Worker  Employee 
Shortage  Report,  A  Civil  Service  Recruitment  and  Retention  Crisis  (Washington,  D.C.:  ASTHO.org),  3. 

20  Federal  Law  Enforcement  Statistics,  Bureau  of  Justice  Statistics,  1 . 

21  Ibid. 
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enforcement.  In  order  to  maximize  our  defense  against  asymmetric  threats,  we  must 
effectively  utilize  unclassified  intelligence  to  educate  these  diverse  professionals  on 
threats,  engage  them  in  the  intelligence  process,  and  enlist  them  to  provide  information  to 
federal  partners  so  that  appropriate  preventative  measures  can  be  considered.  We  must 
improve  our  performance  to  meet  this  need.  Attorney  General  Alberto  Gonzalez, 
speaking  before  the  Council  on  Foreign  Relations  in  New  York,  remarked  on  December 
1,  2005,  “we  must  fight  an  intelligent  war  against  terrorism  using  every  tool 
available. ..This  is  the  government’s  obligation,  and  it  is  the  American  people’s 
expectation...  it  is  your  expectation.”22  If  we  are  to  use  “every  tool  available,”  and  “fight 
an  intelligent  war...”  then  we  must  educate  and  utilize  the  98.5%  of  law  enforcement 
that  are  not  FBI  agents  along  with  other  NTR  professionals  in  the  asymmetric  conflict 
threat  posed  by  hostile  non-state  actors.  If  found  to  be  effective,  CBF  for  NTR  would 
represent  a  concrete,  visible  step  in  transformation  from  a  need-to-know  to  a  need-to- 
share  culture. 

Classified  intelligence  needs  of  local  law  enforcement  are  being  addressed 
through  Joint  Terrorism  Task  Forces  (JTTF)  and  the  development  of  state  and  local 
fusion  centers,  some  of  which  include  fire  service  and  private  sector  partners.  Classified 
and  unclassified  information  can  be  shared  with  participants  who  have  received  security 
clearances.  One  of  the  limitations  of  these  centers  is  the  need  for  those  directly  involved 
with  the  centers  to  live  in  the  geographic  region  of  the  centers;  in  a  state  such  as 
California  that  covers  a  vast  geographic  area,  this  can  be  a  significant  limitation,  even 
with  multiple  fusion  centers  within  the  state.  It  is  clear  that  technological  assistance  will 
be  required  to  reach  a  critical  mass  of  professionals  over  a  wide  area  in  a  timely  manner. 
As  noted  in  the  Markle  report,  “Protecting  America’s  Freedom  in  the  Information  Age,” 

The  DHS  should  become  the  base  for  building  up  a  national  community  of 
intelligence  contributors  and  analysts.  To  create  a  national  infrastructure 
that  is  aware,  robust,  and  resilient  to  the  many  challenges  we  face  in  the 
21st  century,  we  have  to  harness  the  power  and  dynamism  of  information 


22  Alberto  R.  Gonzales,  Prepared  Remarks  for  Attorney  General  Alberto  R.  Gonzales 
(lecture,  Council  on  Foreign  Relations,  New  York,  December  1,  2005),  1, 
http://www.usdoj.gov/ag/speeches/2005/ag_speech_051201.html  (accessed  December  4,  2005). 
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technology  by  utilizing  the  strengths  and  mitigating  the  weaknesses  of  our 
networked  society.23  (Emphasis  Added) 

CBF  intelligence  must  utilize  infonnation  technology  with  an  ability  to  reach  over 
87,000  jurisdictions,  while  also  maintaining  integration  with  other  Federal,  State  and 
Regional  efforts.  The  implications  to  homeland  security  are  significant.  The  adage, 
“Quantity  has  a  quality  all  its  own”  reflects  the  potential  impact  of  the  improved  use  and 
involvement  of  NTR  in  the  intelligence  process.  Small  expenditures  in  improving 
unclassified  intelligence  are  leveraged  by  the  millions  of  diverse  NTR  along  with  the 
billions  of  dollars  expended  to  date  in  community  outreach  programs  such  as  community 
oriented  policing  (COP).  An  estimated  7.5  billion  dollars  was  expended  for  COP  alone 
from  1994-2000.24 

Significant  efforts  have  been  focused  on  intelligence  analysis  and  upward 
intelligence  flow;  stimulating  NTR  involvement  in  intelligence  operations  through 
improved  dissemination  can  exponentially  increase  the  quantity  and  quality  of  that 
upward  intelligence  flow.  Focusing  readily  available  technology,  existing  infrastructure, 
and  intelligence  smart  practices  on  NTR  will  result  in  improved  information  flow, 
supporting  analytical  success  and  initiating  an  upward  spiral  of  intelligence  quality  and 
effective  counter-terrorism  operations.  As  noted  in  the  Department  of  Army 
Counterinsurgency  Manual,  Final  Draft, 

Intelligence  and  operations  feed  back  on  one  another.  Effective 
intelligence  drives  effective  operations,  which  produce  more  intelligence. 
Similarly,  ineffective  or  inaccurate  intelligence  produces  ineffective 
operations,  which  reduce  the  availability  of  intelligence.25 


23  Zoe  Baird,  Protecting  America's  Freedom  in  the  Information  Age  (New  York,  New  York:  The 
Markle  Foundation,  2002),  15,  http://knxup2.ad.nps.navy.mil/homesec/docs/nonprof/nps05-070903-01.pdf 
(accessed  February  2,  2006). 

24  G.  G.  Davis  and  others,  The  Facts  about  COPS:  A  Performance  Overview  of  the  Community’ 
Oriented  Policing  Services  Program  (Washington,  D.C.:  Fleritage  Foundation,  2000),  1, 
http://www.heritage.org/Research/Crime/CDA00-10.cfm  (accessed  January  12,  2007). 

25  Headquarters,  Department  of  the  Army,  COUNTERINSURGENCY  Draft  Not  for  Implementation. 
(Washington,  D.C.:  Headquarters,  Department  of  the  Army,  2006),  3-2, 
http://www.fas.org/irp/doddir/army/fm3-24fd.pdf  (accessed  August  12,  2006). 
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D.  REVIEW  OF  RELEVANT  LITERATURE 


1.  What  is  Contextual  Background? 

In  the  study  of  philosophy,  some  believe  the  understanding  of  text  is  dependent  on 
the  context  in  which  the  reader  exists.  As  the  text  is  read,  it  is  interpreted  based  on  the 
social  context  and  bias  of  the  interpreter.  Additionally,  text  has  its  own  “horizon  of 
meaning”  which  is  influenced  by  the  contextual  background  of  the  writer,  the  time  of 
writing,  and  the  originating  context.  Philosophical  henneneutics  examines  the 
relationship  between  a  reader  and  text,  both  of  which  must  be  understood  within  the 
context  of  their  experience  and  creation  respectively.26 

Hermeneutics  as  a  theory  involves  both  the  understanding  and  interpretation  of 
text  that  has  continued  from  the  time  of  Plato.  It  has  been  commonly  applied  to  both 
Biblical  and  cultural  research.  In  “Teoria  Della  Interpretatione,”  Emilio  Betti  argues  that 
text  is  “objectified  representations  of  human  intentions,”  and  to  grasp  the  true  and  only 
meaning  of  the  text,  a  reader  must  understand  the  original  process  of  creation.27  In  the 
creation  and  distribution  of  unclassified  intelligence,  hermeneutic  fusion  is  problematic; 
the  original  process  of  creation  is  often  based  on  classified  intelligence,  which  may  not  be 
fused  to  an  unclassified  document.  As  the  author/producer  of  unclassified  intelligence  is 
prohibited  from  the  fusion  of  classified  henneneutic  material,  open  source  contextual 
background  chosen  by  authors/producers  must  be  used  to  replace  the  original,  classified 
material  that  influenced  the  hermeneutic  of  the  writer  at  the  time  of  creation.  In  this 
research,  contextual  background  is  defined  as  unclassified  material  that  best  represents 
the  hermeneutic  of  the  producer  at  the  time  of  creation. 

2.  Contextual  Background  and  Intelligence  Usefulness 

Discussions  about  intelligence  usefulness  have  been  focused  primarily  on 
traditional,  classified,  intelligence  recipients  in  the  government.  As  intelligence  agencies 
move  from  a  “right  to  know”  to  a  “right  to  share”  culture  and  more  NTR  begin  to  receive 
unclassified  intelligence  products,  intelligence  producers  must  focus  on  intelligence 

26  Frank  Ravitch,  “Struggling  with  Text  and  Context:  A  Hermeneutic  Approach  to  Interpreting  and 
Realizing  Law  School  Missions,”  St.  Johns  Law  Review  (2000),  5, 

http://fmdarticles.eom/p/artieles/mi_qa3735/is_200007/ai_n8886993/pg_5  (accessed  10,  2006). 

27  Stanford  Encyclopedia  of  Philosophy,  Henneneutics  (Palo  Alto,  CA:  Stanford  Encyclopedia  of 
Philosophy,  2005),  1,  http://plato.stanford.edu/entries/hermeneutics/  (accessed  January  12,  2007). 
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usefulness  for  NTR.  As  expressed  by  John  Hillen  in  the  National  Review,  “U.S. 
Intelligence  Failures  stem  from  too  much  infonnation,  not  enough  understanding.”28 
This  is  a  subtle,  but  critical  point;  even  a  high  quality  intelligence  product  that  is  not  fully 
understood  by  NTR  may  lack  usefulness,  failing  to  provide  the  knowledge  required  to 
improve  decision  making.  There  is  agreement  that  ultimately,  it  is  the  usefulness  of  the 
intelligence  product  in  achieving  homeland  security  that  matters. 

The  sentiment  of  Hillen  is  echoed  in  CIA  reports,  outlining  that  the  provision  of 
contextual  background  is  a  critical  component  in  the  usefulness  of  intelligence  products. 

In  periods  of  crisis,  when  demands  are  high  and  response  time  is  short, 
most  written  intelligence  production  is  in  the  form  of  policy-driven  memos 
and  briefs  and  pieces  written  for  daily  publications.  The  result  of  this 
narrowly  focused  and  piecemeal  intelligence  flow  is  that  it  does  not  foster 
continuity  of  analysis  nor  does  it  provide  a  context  within  which  to  place 
seemingly  unrelated  information.  In  the  case  of  Iraq,  national  intelligence 
did  not  provide  a  comprehensive  picture  of  how  the  country  functioned  as 
a  whole.  The  Intelligence  Community  has  made  substantial,  although 
sporadic,  efforts  over  the  past  decade  and  a  half  to  explore  better  and 
more  technologically  advanced  methods  of  communicating  with 
consumers.  The  results,  however,  have  been  modest  at  best.  The 
requirement  to  have  background  and  contextual  information  available 
at  the  policymaker's  fingertips  in  a  timely  fashion  remains  unfulfilled.29 
(Emphasis  added) 


Decision  makers  often  demand  “tailored”  intelligence  briefs  beyond  what  is 
prepared  for  general  distribution  in  order  to  meet  their  decision  making  needs.30  This 
issue  is  exacerbated  by  the  recent  addition  of  NTR.31  These  recipients  do  not  have 
personal  intelligence  analysts  to  produce  “tailored”  briefs  that  contain  necessary 
contextual  background.  Additionally,  NTR  are  often  unaccustomed  to  the  intelligence 

28  John  Hillen,  “Know  Nothings:  U.S.  Intelligence  Failures  Stem  from  Too  Much  Information,  Not 
enough  Understanding,”  National  Review,  8/  3.  (1998),  1, 

http://www.fmdarticles.eom/p/articles/mi_ml282/is_nl4_v50/ai_21102283  (accessed  January  12,  2007). 

29  Kerr  et  ah.  Intelligence  and  Analysis  on  Iraq:  Issues  for  the  Intelligence  Community,  1, 
http://www.gwu.edu/~nsarchiv/news/20051013/kerr_report.pdf.  (accessed  January  24,  2006). 

30  Peter  Pirolli,  Assisting  People  to  Become  Independent  Learners  in  the  Analysis  of  Intelligence  (Palo 
Alto,  California:  Office  of  Naval  Research,  2006),  38,  http://www.fas.org/irp/eprint/pirolli.pdf.  (accessed 
September  2,  2006). 

3 1  Anonymous,  Northern  California  Regional  Terrorism  Threat  Assessment  Center, 
http://www.ncrttac.org/  (accessed  January  24,  2006),  1. 
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cycle  and  use  of  intelligence  products  as  their  discipline  related  training  and  experience 
did  not  previously  require  the  use  of  such  intelligence.  As  described  by  Lisa  Palmieri, 
President  of  the  International  Association  of  Law  Enforcement  Intelligence  Analysts, 


I  recall  reading  many  articles  in  the  media  in  which  law  enforcement 
executives  demanded  more  “information  sharing”,  particularly  focused  at 
the  FBI.  I  also  recall  thinking  that  if  these  executives  got  what  they  asked 
for,  they  would  be  buried  in  uncorroborated,  unevaluated,  “white 
noise”.  At  that  time,  very  few  state  and  local  police  departments  had 
analytic  capabilities,  so  being  barraged  with  information  would  defeat  the 
purpose  of  guiding  law  enforcement  decisions.  This  has,  unfortunately, 
come  to  pass,  with  law  enforcement  agencies  erring  on  the  side  of  caution; 
they  are  sharing  more  piece-meal  information  than  could  ever  be  made 
useful  in  case  any  small  detail  might  possibly  be  deemed  important  in 
retrospect.  (Emphasis  added)32 

Volumes  of  information  without  contextual  background  can  overwhelm  NTR  and 
is  not  useful. 


3.  Contextual  Background  and  Decision  Making 

There  is  agreement  that  contextual  background  is  a  critical  component  in 
intelligence  decision  making.33  Extensive  research  exists  on  decision  making  theory, 
with  similarly  extensive  discussion  of  the  impact  of  intelligence  products  on  decision 
making.  Researchers  in  one  camp  of  decision  making  theory  define  three  different  types 
of  context  that  are  relevant  to  decision  making: 

•  Proceduralized  context  which  is  shared  by  those  involved  in  the  problem  and 
is  directly  but  tacitly  used  for  the  problem  solving. 

•  Contextual  knowledge  that  is  not  explicitly  used  but  influences  the  problem 
solving. 

•  External  knowledge  that  has  nothing  to  do  with  the  current  decision  making 
but  is  known  by  many  of  those  involved. 


32  Lisa  M.  Palmieri,  Information  vs.  Intelligence:  What  Police  Executives  Need  to  Know 
(Massachusetts:  1ALEIA,  2005),  1,  http://www.ialeia.org/pubs/InformationvsIntelligence.pdf  (accessed 
January  28,  2006). 

33  H.  Bradford  Westerfield,  “Inside  Ivory  Bunkers;  CIA  Analysts  Resist  Managers’  ‘Pandering’,”  in 
Strategic  Intelligence:  Windows  into  a  Secret  World,  ”  ed.  by  Loch  Johnson  and  James  Wirtz  (Los  Angeles, 
California:  Roxbury  Publishing  Company,  2004),  198, 

http://www.hsdl.org/homesec/docs/zroom/cm/ns4156/westerfield2004pl98.pdf  (accessed  February  2, 
2006). 
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Proeeduralized  contexts 


Figure  1.  Different  types  of  context  (From  Brezillon  and  Pomerol,  1999) 

Context  is  defined  as  the  “sum  of  all  the  knowledge  possessed  by  the  operators  on 
the  whole  task.”34 

In  this  Pomerol  and  Brezillon  model,  a  decision  maker  would,  when  presented 
with  an  intelligence  product,  access  his  or  her  contextual  knowledge,  and  then 
proceduralize  that  knowledge  based  on  the  intelligence  product  in  hand;  in  this  process, 
the  decision  maker  may  also  access  available  external  contextual  information, 
incorporating  that  knowledge  and  proceduralizing  it  prior  to  decision  making.  As  noted 
by  Pomerol  and  Brezillon,  “...  it  is  clear  that  a  Palestinian  whose  prior  knowledge  is 
reduced  to  his  Imam’s  preaches  cannot  have  the  same  interpretation  of  Middle  East 
events  as  a  Harvard  alumnus.”35  This  research  addresses  the  impact  of  providing 
external  contextual  knowledge  to  decision  makers. 


34  Patrick  Brezillon  and  Jean-Charles  Pomerol,  Procedwalization  of  the  Contextual  Knowledge  for 
Decision  Making,  Abstract  (October  28,  2002),  3,  http://www- 

poleia.lip6.fr/~brezil/Pages2/Publications/Luxembourg.pdf  (accessed  February  2,  2006). 

35  Patrick  Brezillon  and  Jean-Charles  Pomerol,  Procedwalization  of  the  Contextual  Knowledge  for 
Decision  Making,  Abstract  (October  28,  2002),  3,  http://www- 

poleia.lip6.fr/~brezil/Pages2/Publications/Luxembourg.pdf  (accessed  February  2,  2006). 
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Those  involved  in  the  practical  application  of  decision  making  contend  that 
“contextual  analysis”  of  intelligence  is  vital.  While  many  understand  that  intelligence 
intercepted  “right  from  the  enemy’s  mouth”  is  necessary  for  accuracy,  others  argue  that 
human  intelligence  sources  add  the  required  context  to  most  fully  understand  and  utilize 
information.36  Both  camps  agree  that  context  is  important  in  the  decision  making  process 
regardless  of  the  source  or  nature  of  the  infonnation;  the  practical,  traditional  model 
illustrated  in  the  Figure  below  outlines  the  impact  of  the  addition  of  context  to  both 
knowledge  and  understanding,  impacting  perceptions  and  decisions. 

A  Traditional  Model  for  Decision  Making 


Figure  2.  Context  (From  the  North  Atlantic  Treaty  Organization,  2002) 

Understandably,  military  applications  of  context  in  decision  making  center  on 
pragmatic  application.  Training  material  utilized  by  the  North  Atlantic  Treaty 
Organization  (NATO)  reflects  that  practical  application  of  context  in  decision  making: 
“...knowledge  is  produced  when  information  is  correlated  with  a  model  of  the  world  and 

36  Protecting  America ’s  Freedom  in  the  Information  Age ,  48. 
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the  current  context.”  37  Regardless  of  whether  context  and  decision  making  are  evaluated 
through  the  lens  of  the  academic  or  the  war  fighter,  literature  documents  that  context 
involved  in  the  decision  making  process  has  a  significant  impact  on  the  nature,  quality 
and  effectiveness  of  the  decision.  NTR  of  homeland  security  intelligence  must  have  a 
mechanism  to  “...have  background  and  contextual  information  available  at  the  {their) 
fingerprints  in  a  timely  manner...”  in  order  to  most  effectively  utilize  available 
intelligence.  Contextual  background  is  necessary  so  that  NTR  can  make  infonned 
decisions  on  protective  actions  and  response  plans. 

4.  Non  Traditional  Recipients  and  Contextual  Background 

The  provision  of  contextual  background  for  unclassified  intelligence  supplied  to 
NTR  is  not  well  documented.  While  there  is  varied  discussion  on  the  need  for  contextual 
background  regarding  decision  making,  as  well  as  the  importance  of  context  in  military, 
intelligence  and  policy  making  decisions,  the  full  integration  of  NTR  in  homeland 
security  is  still  in  the  early  stages  of  transfonnation  -  the  provision  of  contextual 
background  for  NTR  is  no  exception.  The  CEO  of  Cisco  systems,  John  Chambers, 
highlighted  this  in  a  letter  on  behalf  of  the  National  Infrastructure  Advisory  Council  to 
President  G.W.  Bush:  “Full  integration:  Private  industry  must  be  fully  integrated  into  the 
Federal  Government’s  Intelligence  Cycle.  The  Federal  Government  must  leam  to  include 
industry  experts  as  domestic  intelligence  assets,  integral  to  improving  infrastructure 
protection,  and  not  just  occasional  customers  of  government  intelligence  products.”38 

Urban  Area  Security  Initiatives  (UASI)  provide  one  solution  in  supplying 
homeland  security  contextual  background  to  NTR.  Some  metropolitan  areas  may  have 
the  capacity  in  their  limited  geographic  area  to  provide  context  and  background  for  their 
intelligence  through  traditional  workplace  education;  however,  this  capacity  quickly 
diminishes  the  farther  the  recipient  is  from  intelligence  providers  in  both  distance  and 
discipline.  Intelligence  support  of  NTR  outside  UASI  regions  remains  a  gap  in  the 
current  solutions  set.  Providing  generalized  intelligence  education  through  the  UASI 

37  North  Atlantic  Treaty  Organization,  Tactical  Decision  Aids  and  Situational  Awareness 
(Amsterdam:  North  Atlantic  Treaty  Organization,  2002),  14, 

http://knxup2.ad.nps.navy.mil/homesec/docs/dtic/ADA400661.pdf  (accessed  January  27,  2006). 

38  Chambers  and  Nye,  Letter  to  George  Bush,  Washington,  D.C.;  National  Infrastructure  Advisory 
Council,  1,  http://www.dhs.gov/interweb/assetlibrary/NIAC_EEIS_Letter_0804.pdf  (accessed  January  24, 
2006). 
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network,  including  planning  and  direction,  collection,  processing,  analysis,  production 
and  discrimination,  begins  the  process  of  integrating  NTR  into  the  intelligence  cycle.39 
As  NTR  receive  intelligence  products  relevant  to  their  geographic  area  and  discipline,  the 
provision  of  relevant  background  information  can  provide  context  and  a  depth  of 
understanding  not  otherwise  readily  available,  providing  decision  makers  with  an 
increased  capacity  to  effectively  utilize  the  intelligence  in  protection  of  critical 
infrastructure  and  provision  of  leads  back  to  the  IC. 

In  addition  to  the  UASI  resources  available  in  some  metropolitan  areas,  law 
enforcement  NTR  have  technological  options  in  obtaining  contextual  background  for 
unclassified  intelligence  products  not  available  to  other  NTR  such  as  the  Regional 
Information  Sharing  System  (RISS)  and  Law  Enforcement  Online  (LEO).  Combined, 
firefighters  and  private  sector  security  guards  alone  account  for  approximately  three 
million  NTR  who  currently  lack  access  to  these  systems.40  While  law  enforcement  may 
employ  these  information  systems,  the  contextual  background  contained  in  these  data 
warehouses  is  not  currently  fused  with  intelligence  products,  requiring  recipients  to 
search  for  information.  Although  more  effective  than  an  open  internet  search  (due  to  the 
validity  an  accuracy  of  the  source  information),  accessing  appropriate  infonnation  is  still 
dependant  on  the  varied  skill,  interest  level  and  time  available  to  each  recipient  -  results 
may  not  reflect  the  hermeneutic  of  the  producing  agency.  This  research  examines  the 
impact  of  CBF  to  intelligence  products  for  the  millions  of  first  responders  who  are 
currently  without  access  to  existing  systems  across  America.  Robert  Steele  highlights  the 
value  of  context  in  the  intelligence  process  for  all  recipients: 

New  Value  is  in  Content  +  Context  +  Speed  The  traditional  craft  of 
intelligence  has  tended  to  fragment  content  from  its  context,  and  be 
largely  oblivious  to  timing.  This  is  true  both  in  the  collection  cycle  and  in 
the  production  cycle.  The  new  craft  of  intelligence  recognizes  that  the 
value  of  any  given  information,  apart  from  its  relevance  to  the  decision  at 
hand,  stems  from  a  combination  of  the  content  in  context,  and  the 


39  Paul  W.  Parfomak,  Guarding  America:  Security  Guards  and  U.S.  Critical  Infrastructure 
Protection  (Washington,  D.C.:  The  Library  of  Congress:  Congressional  Research  Service,  2004),  27, 
http://www.fas.org/sgp/crs/RL32670.pdf  (accessed  January  21,  2006). 

40  Ibid. 
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content  in  time.  Both  collectors  and  producers  of  intelligence  must  be 
acutely  sensitive  to  the  day-to-day  needs  of  their  consumers.47  (Emphasis 
Added) 

Steele  understands  that  the  value  of  intelligence  is  not  based  solely  on  the  sources 
utilized  in  the  intelligence  product,  and  that  “historical  knowledge,”  (referenced  by 
Pomerol  and  Brezillon  as  “contextual  knowledge”)  is  a  critical  factor  in  the  value  of 
intelligence  products.  This  research  identifies  the  impact  of  avoiding  “fracturing  content 
from  its  context,”  by  fusing  content  (existing  intelligence  products)  +  context  (vetted, 
valid  and  accurate)  +  speed  (hyperlink  fusion)  for  non  traditional  recipients.  Speed 
through  hyperlink  fusion  leverages  the  attention  span  of  the  reader,  providing  contextual 
background  at  the  exact  moment  the  issue  is  facing  the  reader. 

While  contextual  background  is  not  specifically  mentioned,  the  Office  of  the 
Director  of  National  Intelligence  is  responsible  for  both  traditional  and  non  traditional 
recipient  intelligence  and  “ensures  that  the  Intelligence  Community  is  integrated, 
focusing  on  the  right  questions,  and  maximizing  the  return  on  taxpayer  dollars  by 
identifying  threats  clearly  so  that  policymakers,  legislators,  military  commanders,  and 
law  enforcement  officials  can  make  well-informed  decisions  and  take  effective  actions ” 
(emphasis  added).42  The  National  Strategy  for  Homeland  Security  also  provides 
evidence  and  summarizes  the  significance  of  providing  high  quality  intelligence  to  NTR, 
citing  a  need  to  “Enable  effective  partnership  with  state  and  local  governments  and  the 
private  sector... which  controls  eighty-five  percent  of  America’s  infrastructure... 
Government  at  all  levels  must  enable,  not  inhibit,  the  private  sector’s  ability  to  carry  out 
its  protection  responsibilities.  The  Nation’s  infrastructure  protection  effort  must  harness 
the  capabilities  of  the  private  sector  to  achieve  a  prudent  level  of  security  without 
hindering  productivity,  trade,  or  economic  growth...43  Contextual  background  is  a 
necessary  to  meet  these  mandates. 

41  Robert  Steele,  The  New  Craft  of  Intelligence  Personal,  Public  and  Political  (Oakton,  Virginia:  OSS 
International  Press,  2002),  147-161. 

42  Director  of  National  Intelligence,  Report  on  the  Progress  of  the  Director  of  National  Intelligence  in 
Implementing  the  Intelligence  Reform  and  Terrorism  Prevention  Act  of 2004,  1, 
http://www.fas.org/irp/dni/implement.html  (accessed  January  12,  2007). 

43  George  W.  Bush,  National  Strategy’ for  Homeland  Security’  (Washington,  D.C.:  Office  of 
Homeland  Security,  2002),  5-6. 
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In  May,  2006  the  Director  of  National  Intelligence  (DNI)  affirmed  the  importance 
of  using  data  warehousing  to  store  intelligence  contextual  background  when  he  reported 
to  Congress  on  current  ODNI  efforts  to  establish  digital  libraries  of  all  new  intelligence 
products.  He  also  outlined  efforts  to  improve  the  ability  of  analysts  to  access  all  relevant 
information  for  their  analysis,  both  open  source  and  classified,  regardless  of  original 
author.44  It  is  critical  that  analysts  have  access  to  this  data  warehouse  (library)  of  topical 
intelligence  products  in  order  to  receive  the  contextual  background  necessary  for  them  to 
produce  contextually  accurate  intelligence  products.  It  is  similarly  important  for  NTR  to 
have  a  data  warehouse  (library)  of  contextual  background  for  intelligence  products  so  that 
they  may  produce  better  decisions.  DHS  has  funded  open  source  data  warehouses  such 
as  the  Memorial  Institute  for  the  Prevention  of  Terrorism  (MIPT)  Terrorism  Knowledge 
Base  (TKB)  database  to  address  this  need.45  Existing  investment  in  these  data 
warehouses  can  be  leveraged,  they  provide  a  readily  available,  government  funded,  vetted 
source  of  contextual  background  that  can  be  fused  to  intelligence  products.  Private  sector 
assets  and  government  funded,  open  source  contextual  background  data  warehouses  are 
two  of  many  valuable  assets  that  must  be  fully  integrated  into  the  intelligence  cycle. 

The  research  to  date  provides  valuable  insight  into  the  importance  of  contextual 
background,  the  importance  of  non  traditional  recipients,  and  the  importance  of 
intelligence  in  decision  making.  As  the  provision  of  unclassified  intelligence  to  NTR  is 
relatively  new,  there  is  little  direct  research  on  the  impact  of  context  in  this  area;  this 
impact  must  be  implied  from  experience  with  traditional  intelligence  recipients  combined 
with  the  impact  of  context  in  the  decision  making  of  NTR  in  areas  other  than  intelligence. 

5.  Technology  and  Contextual  Background 

The  literature  shows  that  decision  making  is  impaired  when  insufficient 
information  is  available;  it  also  documents  that  too  much  information  (or  information 
overload)  impairs  decision-making.  Technology  has  the  potential  to  impair  decision 
making  through  the  provision  of  too  much  information,  the  “white  noise”  discussed  by 
Lisa  Palmieri,  or  to  assist  by  increasing  a  decision  maker’s  ability  to  acquire,  transform 

44  Report  on  the  Progress  of  the  Director  of  National  Intelligence  in  Implementing  the  Intelligence 
Reform  and  Terrorism  Prevention  Act  of 2004,  7. 

45  MIPT,  About  MIPT  Terrorism  Knowledge  Base,  Memorial  Institute  for  the  Prevention  of 
Terrorism,  1,  http://www.tkb.org/AboutTKB.jsp  (accessed  September  3,  2006). 
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and  explore  knowledge  as  envisioned  by  the  IC.  Technological  assistance  in  providing 
the  right  amount  and  type  of  knowledge  to  decision  makers  has  the  potential  to  improve 
decisions,  increase  decision  timeliness,  and  decrease  staff  support  requirements.  Given 
the  lack  of  intelligence  specialists  in  NTR  disciplines  outside  of  law  enforcement,  and  the 
fact  that  79%  of  police  departments  have  twenty-five  or  fewer  sworn  officers,  technology 
support  is  critical  if  we  are  to  effectively  engage  NTR  in  homeland  security.46 

A  review  of  knowledge  management  literature  shows  design  parallels  between 
“expert  systems”  and  the  fusion  center/Joint  Terrorism  Task  Force  process.  Fusion 
Centers  and  Joint  Terrorism  Task  Forces  act  as  knowledge  management  engines, 
managing  the  interface  between  NTR,  the  national  knowledge  base,  and  regional 
knowledge  contributed  by  end  users.  Software  engineers  utilize  the  concept  of  a  “context 
manager”  in  designing  automated  software  which  links  disparate  sources  of  data.  Some 
believe  that,  “...the  ability  to  represent  and  manipulate  context  will  be  an  extremely 
important  part  of  providing  semantic  integration  in  multi-database  systems.”47 

Intelligence  products  with  CBF  can  function  as  a  critical  hub  or  node  in 
networking  multiple  databases  of  open  source  contextual  background  and  NTR;  the 
producer  (expert)  manages  the  interface  between  the  warehouses  and  NTR  through  the 
selection  of  appropriate  hyperlinks.  Hyperlinks  have  already  been  utilized  by  some 
unclassified  intelligence  providers,  such  as  the  Illinois  State  Terrorism  Information 
Center  (STIC)  to  provide  contextual  background.48  The  diagram  below  outlines  the 
impact  of  such  hyperlinks  on  a  small  part  of  the  homeland  security  infonnation  network. 


46  Geoffrey  Alpert,  Police  Pursuit  and  the  Use  of  Force.  A  Final  Report  to  the  National  Institute  of 
Justice  (Washington,  D.C.:  National  Criminal  Justice  Reference  Service,  [1997]),  7, 
http://www.ncjrs.gov/pdffiles/164831.pdf  (accessed  May  25,  2006). 

47  Zhengxin  Chen,  Computational  Intelligence  for  Decision  Support  (Boca  Raton,  FL:  CRC  Press, 
2000),  27,  http://www.engnetbase.com.libproxy.nps.navy.mil/books/482/1799_PDF_TOC.pdf  (accessed 
May  26,  2006). 

48  Kevin  Eack  (Illinois  State  Police,  State  Terrorism  Intelligence  Center),  interview  by  author, 
Springfield,  Illinois,  June  18,  2006. 
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Figure  3.  Homeland  security  information  source  network 

Understanding  the  critical  importance  of  improving  the  usefulness  of  intelligence 
products  for  decision  makers,  the  IC  has  utilized  varied  methods  to  provide  user  defined 
access  to  contextual  background  for  intelligence  products,  including  personal  briefings  by 

intelligence  professionals,  ongoing  feedback  to  producers  on  provided  intelligence 
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products  as  part  of  the  intelligence  cycle,  along  with  technological  solutions  such  as  the 
maintenance  of  searchable  data  warehouses  of  contextual  background  and  a  finished 
classified  intelligence  dissemination  system  (Intelink).  The  experience  with  Intelink 
highlights  the  importance  of  technology  acceptance  by  decision  makers;  Intelink  is  a 
secure  network  of  intelligence  databases  that  supports  user  defined  receipt  of  customized 
intelligence  products.49  It  is  intended  to  “provide  robust  and  timely  access  to  all 
available  intelligence  information,  regardless  of  location,  medium,  or  fonnat,  for  all 
interested  users  ...  who  are  authorized  access.”50  Incorporating  collaborative  computing 
on  this  secure  network,  recipients  are  able  to  teleconference  and  whiteboard  with  live 
audio,  video,  document,  and  application  share  capability  along  with  simultaneous  access 
to  multiple  intelligence  data  warehouses.  Intelink  was  designed  not  to  just  “push” 
intelligence  information;  it  allows  recipients  to  “pull”  information  as  well.51 

Intelink  has  not  been  as  successful  as  envisioned.  Many  intelligence  recipients 
have  expressed  a  preference  for  hard  copies  of  reports,  personal  briefings  and  traditional 
communication  methods  over  Intelink.52  As  reflected  in  Technology  Acceptance  Model 
research,  ease  of  use  and  perceived  usefulness  are  significant  factors  in  the  successful 
implementation  of  any  solution.  Despite  the  extensive  capabilities  of  Intelink,  users 
“claim  they  go  first  to  the  Agency  web  sites,  find  no  information  at  all,  usually  become 
quickly  frustrated,  and  log  off  with  the  impression  the  intelligence  agencies  do  not  store 
information  on  Intelink.”53  Decision  makers  want  a  summary  of  information  the  briefer 
believes  is  important  (push)  and  want  additional  information  the  decision  maker  believes 
may  be  important  from  the  briefer  immediately  thereafter  (pull).  Intelligence  system 

49  Lome  Teitelbaum,  The  Impact  of  the  Information  Revolution  on  Policymakers'  use  of  Intelligence 
Analysis  (Santa  Monica,  California:  RAND,  2005),  98, 

http://www.rand.org/pubs/rgs_dissertations/2005/RAND_RGSD186.pdf  (accessed  May  24,  2006). 

50  Fredrick  Martin,  Top  Secret  Intranet:  How  U.S.  Intelligence  Built  Intelink  -  the  Worlds  Largest 
most  Secure  Network  ( Upper  Saddle  River,  NJ:  Prentice  Hall,  1999),  1-380. 

51  Patrick  Kelly,  Intelligence  Support  to  Homeland  Security’:  Supporting  the  Supporting  Effort 
(Carlisle  Barracks,  Pennsylvania:  U.S.  Army  War  College,  2002),  29, 
https://www.hsdl.org/homesec/docs/dtic/ADA404363.pdf  (accessed  May  21,  2006). 

52  Teitelbaum,  The  Impact  of  the  Information  Revolution  on  Policymakers'  use  of  Intelligence 
Analysis,  101. 

53  Ibid.,  106. 
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design  must  support  the  needs  and  preferences  of  decision-makers  or  run  the  risk  of 
rejection,  regardless  of  how  exceptional  the  technology  or  the  potential  of  the  system. 

6.  Technology  Acceptance 

The  theory  of  reasoned  actions  (TRA)  serves  as  a  theoretical  base  for  examining 
technology  acceptance.54  TRA  posits  that  an  individual’s  beliefs  influence  their  attitudes, 
that  when  combined  with  societal  norms,  drives  behavioral  intentions,  leading  to  actual 
behavior.  Based  on  TRA,  the  technology  acceptance  model  (TAM)  is  an  established 
method  of  predicting  user  acceptance.  In  this  model,  perceived  “Ease  of  Use”  and 
perceived  “Usefulness”  explain  why  individuals  accept  or  do  not  accept  technology.55  A 
review  of  previous  studies  shows  that  TAM,  with  strong  empirical  support,  has  become  a 
dominant  model  for  predicting  technology  adoption.56  TAM  is  one  method  to  predict 
user  acceptance  before  large  scale  investment  or  commitment  to  a  technology  in  mission 
critical  systems. 

Perceived  ease  of  use  is  the  degree  to  which  a  user  believes  that  using  a 
technology  would  be  effortless,  and  perceived  usefulness  is  the  degree  to  which  a  user 
believes  that  a  technology  would  improve  perfonnance,  production  or  effectiveness.  Of 
the  two  constructs,  it  appears  that  usefulness  is  critical,  “no  amount  of  ease  of  use  can 
compensate  for  a  system  that  does  not  perform  a  useful  function.”57  The  use  of 
technology  in  knowledge  management  systems  has  been  studied  using  TAM  to  determine 
factors  that  impact  loyal  use;  both  perceived  usefulness  and  ease  of  use  were  found  to  be 
factors  positively  related  to  loyal  use.58 

54  M.  Fishbein  and  I.  Ajzen,  Belief,  Attitude,  Intention,  and  Behavior:  An  Introduction  to  Theory’  and 
Research  (MA:  Addison-Wesley  Reading,  MA,  1975),  21-52. 

55  F.  D.  Davis,  R.  P.  Bagozzi  and  P.  R.  Warshaw,  User  Acceptance  of  Computer  Technology’:  A 
Comparison  of  Two  Theoretical  Models,  Management  Science  35,  no.  8  (1989),  982-1003. 

56  Chienting  Lin  and  others,  Examining  User  Acceptance  of  COPLINK  Technologies  by  Law 
Enforcement  Officers:  A  Survey  Study  (Arizona:  University  of  Arizona, 2002),  2, 

http://citeseer.ist.psu.edU/cache/papers/cs2/365/http:zSzzSzwww.digitalgovernment.orgzSzlibraryzSzlibrar 
yzSzdgo2002zSz..zSzpdfzSzlin.pdf/examining-user-acceptance-of.pdf  (accessed  August  20,  2006). 

57  F.  D.  Davis,  R.  P.  Bagozzi  and  P.  R.  Warshaw,  “User  Acceptance  of  Computer  Technology:  A 
Comparison  of  Two  Theoretical  Models,”  Management  Science  35,  no.  8  (1989),  333. 

58  Paul  Clay,  Alan  Dennis  and  Dong-Gil  Ko,  Factors  Affecting  the  Loyal  use  of  Knowledge 
Management  Systems  (Indiana:  Indiana  University, 2005),  1-10, 
http://csdl2.computer.org/comp/proceedings/hicss/2005/2268/08/22680251c.pdf. 
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E.  RESEARCH  QUESTIONS 

•  Does  CBF  increase  customer  satisfaction  “value”  for  NTR  as  defined  by 
DHS/FBI? 

•  Does  CBF  increase  customer  satisfaction  “quality”  for  NTR  as  defined  by 
DHS/FBI? 

•  Will  NTR  find  hyperlink  technology  applications  in  unclassified  intelligence  as 
“easy  to  use”  as  defined  by  technology  acceptance  model  research? 

•  Will  NTR  perceive  hyperlink  technology  applications  in  unclassified  intelligence 
as  “useful”  as  defined  by  technology  acceptance  model  research? 

•  Given  a  choice,  will  NTR  believe  a  CBF  intelligence  document  is  of  greater  value 
to  themselves  and  their  organization  than  a  non  CBF  product? 

F.  CHAPTERS 

This  research  reviews  hermeneutic  theories,  prior  studies  on  the  impact  of 
contextual  background  in  decision  making,  current  unclassified  intelligence  distribution, 
and  technology  acceptance  research.  The  acceptance  of  hyperlink  technology  to  achieve 
CBF  is  evaluated  utilizing  previously  validated  technology  acceptance  model  factors  of 
ease  of  use  and  usefulness.  The  impact  of  CBF  on  value  and  quality  factors  is 
independently  tested  using  measures  defined  by  DHS  and  FBI. 

Chapter  I  serves  as  an  introduction  and  summary  then  presents  important 
background  and  context  to  this  research,  reviewing  relevant  literature  on  the  impact  of 
context  in  decision  making,  technology  acceptance,  and  the  direction  set  by  senior  policy 
makers  in  intelligence  distribution.  The  significant  citations  of  senior  policy  makers, 
researchers  and  practitioners  leads  to  the  conclusion  that  homeland  security  decision 
makers  require  contextual  background  for  intelligence  products  in  order  to  best  use  that 
intelligence. 


23 


Chapter  II  reviews  the  existing  provision  of  intelligence  products  in  context  with 
an  emphasis  on  unclassified  material.  Multiple  strategies  in  local,  regional,  state,  federal 
and  international  agencies  are  reviewed. 

Chapter  III  establishes  hypothesis’  and  outlines  the  process  of  building,  testing, 
and  validating  the  testing  process,  leading  to  the  results  of  that  testing.  Subject  matter 
experts  were  consulted  at  every  step  of  the  process  to  ensure  both  validity  and  practical 
application.  With  the  input  of  subject  matter  experts  (SME),  the  goal  of  conducting 
valid,  scholarly  research  that  improves  perfonnance  in  the  GWOT  was  paramount.  The 
implementation  of  well  intentioned,  reasonable  strategies  and  tactics  is  often  necessary 
without  the  benefit  of  validated  research,  CHDS  provides  both  the  mandate  and 
opportunity  to  research  and  validate  both  strategies  and  tactics. 

The  final  chapter  presents  recommendations  based  on  the  statements  of  senior 
policy  makers,  researchers  and  practitioners,  combined  with  feedback  from  NTR 
presented  with  both  traditional  and  CBF  products. 


24 


II.  EXISTING  PROVISION  OF  INTELLIGENCE  PRODUCTS  IN 

CONTEXT 


The  provision  of  contextual  background  in  intelligence  can  be  described  as  “the 
paradox  of  plenty.”  Although  context  is  available  in  many  open  source  Internet 
accessible  databases  that  are  relevant  to  intelligence  products  distributed  to  NTR,  the 
distributed  data  warehouses,  lack  of  linking  between  warehouses  and  intelligence 
products,  lack  of  reconciliation  with  intelligence  products,  along  with  the  sheer  volume  of 
context  interferes  with  user  access  to  this  “plenty. ”59 

A  RAND  study  conducted  for  the  FBI  summarized  the  importance  of  providing 
appropriate  intelligence  products  to  NTR.  This  report  indicated  that  a  review  of  best 
practices  of  multiple  country  threat  assessments  demonstrated  a  need  for  products  to  be: 

•  Tailored  to  meet  requests  from  government  departments,  police,  and 
private  industry. 

•  Frequently  incorporate  a  process  for  sending  “scrubbed”  versions  to  non- 
cleared  customers.60 

“Tailoring”  intelligence  for  diverse  recipients  cuts  to  the  heart  of  NTR  need  for 
different  levels  of  contextual  background.  While  historically  recipients  may  have  been 
relatively  homogenous  in  discipline  and  hermeneutic  understanding  of  intelligence 
products,  the  current  diversity  of  disciplines  and  contextual  knowledge  of  NTR  ensures 
that  “one  size  fits  all”  unclassified  intelligence  products  will  fit  almost  no  one. 


A.  DEPARTMENT  OF  THE  ARMY 

In  Police  Intelligence,  the  United  States  Military  uses  hyperlinks  to  individually 
“tailor”  unclassified  intelligence  products.  An  example  of  this  networking  of  open  source 
material  to  unclassified  intelligence  documents  is  demonstrated  in  the  July  2006  Police 


59  Nazli  Choucri,  Stuart  Madnick  and  Michael  Siegel,  Improving  National  and  Homeland  Security 
through  Context  Knowledge  Representation  and  Reasoning  Technologie.  (Working  Paper,  Composite 
Information  Systems  Laboratory,  Sloan  School  of  Management,  Massachusetts  Institute  of  Technology, 
Cambridge,  MA,  2006),  1-25,  http://web.mit.edu/smadnick/www/wp/2006-03.pdf  (accessed  August  3, 
2006). 

60  Peter  Chalk,  Seth  Jones  and  William  Rosenau,  Understanding  the  Enemy:  Lessons  for  Effective 
Threat  Assessments  (Santa  Monica,  CA:  RAND,  2004),  9. 
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Intelligence  Operations  Field  Manual.61  Hyperlinks  to  open  source  websites  such  as  FBI, 
news  agencies,  and  the  Postal  Service  are  used  as  examples  of  fusing  unclassified 
intelligence  products  with  open  source  material  in  a  sample  unclassified  intelligence 
product.  As  these  hyperlinks  are  standard,  “ease  of  use”  should  be  same  as  the 
experimental  product  in  this  research.  This  use  of  hyperlink  technology  to  allow  a  “semi¬ 
custom”  user  defined  experience  within  the  confines  of  open  source  links  provided  by  the 
analyst  is  one  method  to  meet  the  recommendations  of  RAND,  given  diverse  individual 
recipient  needs  across  a  wide  geographic  area. 

The  U.S.  Military  is  not  unique  in  its  use  of  hyperlinks  to  provide  a  “semi¬ 
custom”  unclassified  intelligence  product  for  diverse  recipients.  Multiple  domestic 
intelligence  fusion  centers  have  used  the  same  strategy.  The  U.S.  Capital  Regional 
Intelligence  Center  utilizes  hyperlinks  in  footnote  citations  in  For  Official  Use  Only 
(FOUO)  Information  Bulletins.  These  footnotes  parallel  the  Military  Police  Intelligence 
Operations  Field  Manual,  directing  interested  recipients  to  open  source  material, 
primarily  news  reports.62  Generally,  these  hyperlinks  are  to  original  source  material  of 
articles  summarized  in  the  report,  rather  than  to  contextual  background. 


B.  NEWS  AGENCY 

An  example  of  CBF  for  a  diverse  audience  can  be  found  at  the  website  of 
television  producer  “FOX  NEWS.”  On  the  FOX  NEWS,  Homeland  Security,  Weapons 
of  Mass  Destruction  website,  hyperlinks  take  interested  readers  to  contextual  background 
on  such  keywords  as  “anthrax,”  “dirty  bomb,”  and  “e  bomb.”  Additionally,  a 
“background”  box  is  imbedded  in  the  article  with  hyperlinks  to  additional  reliable  open 
source  data  warehouses  such  as  “RAND,”  “ready.gov,”  and  “U.S.  Army  Institute  of 
Infectious  Disease. ”63  This  is  an  example  of  CBF  utilizing  accurate,  reliable  and  vetted 

61  Department  of  the  Army,  Police  Intelligence  Operations  (Washington,  D.C.:  Headquarters, 
Department  of  the  Army,  2006),  B-l,  http://www.fas.org/irp/doddir/army/fm3-19-50.pdf  (accessed  August 
4,  2006). 

62  National  Capital  Regional  Information  Center  Information  Bulletin  IB-06-04,  June  30,  2006. 

63  Liza  Porteus,  Weapons  of  Mass  Destruction  Handbook,  Fox  News,  1, 

http  ://www.foxnews. com/story/0, 2933, 76887, 00. html?sPage=fnc.specialsections/homelandsecurity 
(accessed  September  10,  2006). 
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government  sources  for  a  diverse  audience  that  is  examined  in  this  research.  The 
“usefulness”  of  CBF  in  these  products  should  be  similar  to  the  usefulness  factor 
examined  in  this  research. 

C.  DHS/FBI 

Standard  joint  FBI/DHS  unclassified  intelligence  products  for  law  enforcement 
are  distributed  in  PDF  format,  generally  without  hyperlink  or  other  networking  with 
contextual  background.  These  products  are  distributed  to  police,  fire,  EMS,  Public 
Health  and  private  sector  recipients  on  a  regular  basis.64  In  this  “one  size  fits  all” 
distribution,  the  analyst  must  include  contextual  background  they  believe  is  required  by 
the  diverse  recipients  without  making  the  product  so  long  that  it  will  not  be  read.  Each  of 
these  products  contains  a  customer  satisfaction  survey  that  measures  the  value  and  quality 
factors  utilized  in  this  research. 

Internally,  the  Department  of  Homeland  Security  has  the  capacity  for  automated 
contextual  linking  within  the  twenty-one  agencies  in  the  department  through  250,000 
“Autonomy”  software  seat  licenses.  According  to  Autonomy  co-founder  Richard  Gaunt, 
“Autonomy’s  intelligence  data  operating  layer  (IDOL)  provides  that  context,  as  well  as 
the  ability  to  bring  together  relevant  information  from  disparate  sources.”65 

The  FBI  provides  both  unclassified  and  classified  intelligence  to  law  enforcement 
through  4,000  officers  and  agents  assigned  to  a  network  of  101  Joint  Terrorism  Task 
Forces  (JTTF)  linked  to  each  FBI  field  office.66  Supported  by  FBI  Field  Intelligence 


64  Northern  California  Regional  Terrorism  Threat  Assessment  Center,  Northern  California  Regional 
Terrorism  Threat  Assessment  Center,  A  Local,  State  and  Federal  Partnership  with  the  San  Francisco  FBI, 
FIG,  and  JTTF  (San  Francisco;  Northern  California  Regional  Terrorism  Threat  Assessment  Center,  2006), 
1. 


65  Autonomy  Corporation  pic,  Industry  Analyst  Research  Highlights  Autonomy's  Technology’  Over 
Legacy  Rules-Based  Only  Approach  (Sunnyvale,  CA:  Autonomy,  2006),  1, 
http://www.autonomy.com/content/News/Releases/2002/1104.html  (accessed  August  6,  2006). 

66  U.S.  Department  of  Justice,  Fact  Sheet:  Department  of  Justice  Anti-Terrorism  Efforts  since  11 
September  2001,  Release  No.  202-514-2007  (Washington,  D.C.,  2006),  1, 
http://www.usdoj.gov/opa/pr/2006/September/06_opa_590.html  (accessed  January  12,  2007). 
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Groups  (FIG),  local  law  enforcement  liaisons  are  provided  security  clearances  and 
encouraged  to  participate  either  full  time  or  part  time  in  task  force  operations,  providing  a 
layer  of  geographically  tailored  intelligence. 


D.  FUSION  CENTERS 

A  growing  network  of  DHS  supported  state  and  local  fusion  centers  provide  a  link 
between  intelligence  agencies  and  homeland  security  professionals,  either  scrubbing 
classified  intelligence,  producing  independent  intelligence,  or  passing  through 
unclassified  FBI/DHS  products  to  their  geographic  area.  These  centers,  with  diverse 
leadership  structures  and  discipline  participation,  represent  one  method  of  providing 
“tailored”  intelligence  that  a  limited  geographic  area  might  need.  Just  as  the  military 
hyperlink  model  provides  an  individualized  semi-custom  user  defined  experience  across  a 
wide  geographic  area,  the  fusion  center  network  makes  possible  the  customization  of 
intelligence  for  a  specific  geographic  region.  Some  fusion  centers,  such  as  the  Illinois 
Statewide  Terrorism  Intelligence  Center  have  utilized  hyperlinks  to  fuse  contextual 
background  to  their  intelligence  reports  in  efforts  to  improve  intelligence  usefulness. 


E.  NEW  YORK  POLICE  DEPARTMENT 

The  New  York  Police  Department  is  unique  among  the  approximately  18,000 
U.S.  law  enforcement  agencies  due  not  only  to  its  experience  with  terrorism  at  the  World 
Trade  Center,  but  the  sheer  size  of  the  department,  its  responsibilities,  and  its  resources. 
Senior  Advisor  Brian  Jenkins  from  the  RAND  Corporation  summarized  that  “They’re 
developing  best  practices  here  that  should  be  embraced  across  the  country.  The  Feds 
could  leam  from  them.”67  Their  “NYPD  Shield”  program  is  a  continuation  of  New  York 
public/private  partnership,  with  the  department  emailing  unclassified  intelligence 
products  to  private  sector  partners  who  request  participation.  The  “Shield”  intelligence 
products  are  currently  issued  as  PowerPoint  presentations.  These  presentations  include 
significant  contextual  background  including  photographs  and  maps.  On  the  companion 

67  Brian  Jenkins  (RAND  Corporation),  Interview  with  the  National  Tactical  Officers  Association, 
February  2005,  58-71,  http://www.newyorker.com/fact/content/articles/050725fa_fact2. 
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password  protected  website,  access  to  contextual  background  including  links  to  the  MIPT 
Terrorism  Knowledge  Base,  Terrorism  Timelines,  an  archive  of  previous  intelligence 
products,  and  a  small  library  of  terrorism  security  articles  is  available.  These  unclassified 
intelligence  PowerPoint  reports  are  being  secondarily  distributed;  on  August  17,  2006, 
the  California  Office  of  Emergency  Services  redistributed  the  NYPD  Shield  Unclassified 
Mumbai  PowerPoint  to  California  Fire  Chiefs  with  the  comments  “Interesting 
information,  don't  know  if  you  have  seen  it... the  attached  PowerPoint  presentation  is  an 
excellent  overview  of  the  Mumbai  rail  attacks  of  July  1 1,  2006...”  Secondary  distribution 
and  expert  review,  while  antidotal,  indicate  superior  NYPD  product  usefulness. 


F.  DEPARTMENT  OF  DEFENSE 

The  Department  of  Defense  (DoD)  utilizes  Verity/ Autonomy  “K2”  and  “Profiler” 
software  to  provide  internal  context  to  users  of  some  intelligence  systems.  This  software 
performs  multiple  contextual  background  functions.  Upon  data  entry,  a  contextual 
“frame”  is  built  around  the  data  based  on  the  totality  of  the  data  being  entered  and  the 
profile  of  the  submitter;  in  essence,  the  software  attaches  queries  that  match  the  data  at 
the  time  of  data  entry  based  on  the  context  of  the  entry.  This  system,  utilizing  original 
source  data,  comes  closer  to  providing  a  hermeneutic  understanding  of  intelligence; 
intelligence  producers  are  not  required  to  substitute  open  source  material  in  place  of 
classified  original  sources.  Users  of  the  system  build  a  profile  of  themselves,  building 
communities  of  interest  that  the  software  uses  to  prioritize  system  inquiry  responses.  The 
software  builds  upon  that  profile  based  on  entry  and  query  history,  subsequently  using 
this  contextual  base  to  not  only  prioritize  information  that  a  user  attempts  to  “pull”  from 
the  system,  but  to  proactively  create  reports  based  on  the  user’s  contextual  profile.  It 
“pushes”  reports  based  on  new  data  entered  into  the  system  that  matches  the  self- 
described  and  system  use  contextual  profile.  Intelligence  “pulled”  from  and  “pushed”  by 
the  system  is  provided  in  context  based  on  these  “explicit”  and  “implicit”  profiles  of 
users.68  This  software  appears  to  be  designed  primarily  for  analyst  use  rather  than  for 
end  users  of  intelligence. 

68  Christian  Scott  (Autonomy  Software  DHS  Program  Manager),  interview  by  author,  Sunnyvale, 
California,  August  14,  2006. 
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G. 


INTERPOL 


Interpol,  the  international  organization  of  184  member  countries,  has  several 
methods  of  disseminating  terrorism  information  to  member  countries  including  working 
meetings,  warning  lists,  analytical  reports,  an  indices  system,  “orange”  notices,  and 
special  notices.69  Interpol  does  not  generally  receive  classified  intelligence,  but  rather 
intelligence  at  the  Law  Enforcement  Sensitive  level,  therefore,  there  is  no  need  to 
declassify,  amend  or  “scrub”  intelligence  that  is  received  from  members.  Any  member 
country  can  tell  Interpol  who  may  see  information  that  they  provide  to  Interpol,  in 
essence,  the  originating  member  country  remains  the  proprietary  owner  of  information.70 
Interpol’s  I  24/7  virtual  private  network  connects  all  member  countries  and  is  used  to 
distribute  the  law  enforcement  sensitive  intelligence.  This  is  a  law  enforcement  only 
system  that  by  policy  does  not  include  fire,  private  sector,  public  health,  or  emergency 
management.  Information  generally  stays  within  law  enforcement;  however,  it  is 
possible  for  member  agencies  to  share  outside  of  law  enforcement  if  there  is  a 
demonstrated  need,  with  the  permission  of  the  originating  member.  Art  theft  infonnation 
is  one  routine  case  where  information  is  shared  outside  law  enforcement,  being  routinely 
distributed  to  art  dealers  in  the  private  sector  in  order  to  recover  stolen  pieces.71  The 
Interpol  I  24/7  network  does  not  provide  hyperlink  or  other  networking  to  context  or 
background  information,  acting  instead  as  a  distribution  system  for  the  unclassified 
intelligence  products  of  members.72  In  a  manner  similar  to  DHS/FBI  PDF  files, 
intelligence  producers  must  include  contextual  background  as  they  feel  appropriate. 
Unlike  DHS/FBI  products,  Interpol  products  are  generally  distributed  to  only  one 
discipline,  law  enforcement. 


69  INTERPOL,  INTERPOL  Notices.  (Lyon,  Fiance:  INTERPOL,  2006),  1-2, 
http://www.interpol.int/Public/ICPO/FactSheets/GI02.pdf  (accessed  July  27,  2006). 

70  Public  Affairs  Officer  (Public  Affairs  Office,  U.S.  Justice  Department),  interview  by  author, 
Washington,  D.C.,  July  5,  2006. 

71  Public  Affairs  Officer  (Public  Affairs  Office,  U.S.  Justice  Department),  interview  by  author, 
Washington,  D.C.,  July  1,  2006. 

72  INTERPOL,  Stolen  Works  of  Art  (Lyon,  France:  INTERPOL,  2006). 
http://www.interpol.int/Public/WorkOfArt/Default.asp(accessed  August  8,  2006),  1. 
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H.  JANE’S  INFORMATION  GROUP  ONLINE 

Jane’s  Infonnation  Group  Online  is  a  subscription  based  Internet  resource  that 
describes  itself  as: 

...a  world  leading  provider  of  intelligence  and  analysis  on  national  and 
international  defense,  security  and  risk  developments.  Jane’s  is  an 
independent  organization  with  an  unrivalled  reputation  of  accuracy, 
authority  and  impartiality.  Jane’s  delivers  partners  and  clients  a  strategic 
advantage  from  intelligence  acquired  by  a  unique  worldwide  network  of 
independent  analysts.  Governments,  militaries,  business  leaders  and 
academics  in  over  180  countries  rely  on  Jane’s  providing  timely  and 
insightful  information  on  threat  and  security  issues. 

Founded  in  1898,  Jane’s  has  always  been  at  the  forefront  of  publishing 
and  developing  infonnation  solutions  for  its  customers.  Jane’s  provides  a 
unique  and  independent  service  that  allows  its  clients  the  ability  to  make 
mission  or  business  critical  decisions  with  assurance.  Jane’s  understands 
the  importance  of  accurate  infonnation  to  navigate  business  and 
governmental  leaders  through  an  increasingly  turbulent  world.  Every  day 
crucial  decisions  are  made  by  Jane’s  customers  often  based  purely  on 
Jane’s  infonnation.73 

In  a  manner  similar  to  Fox  News,  Jane’s  utilizes  CBF  through  the  use  of 
hyperlinks  to  both  proprietary  and  open  source  government  material  in  their  intelligence 
analysis.  In  addition  to  internal  links,  they  utilize  a  “compendium  of  links  to  government 
information  sites,  counter-terrorism  legislation,  reports  and  position  papers...  and  best 
practice  guidance. .  .”74  In  this  research  the  type  of  CBF  utilized  by  Fox  News  and  Jane’s 
is  evaluated  for  ease  of  use,  usefulness,  value  and  quality  for  application  to  unclassified 
intelligence  distribution  to  NTR. 


I.  CONCLUSIONS 

Because  the  practice  of  providing  unclassified  intelligence  to  NTR  is  relatively 
new,  IC  provision  of  contextual  background  trails  private  sector  “smart  practices.” 
Private  sector  producers  such  as  Fox  News  and  Jane’s  have  significant  experience  in 
producing  reports  for  a  broad  and  diverse  audience,  while  the  IC  has  traditionally 

73  Jane’s  Online,  Jane ’s  Intelligence  and  insight  you  can  trust  (Alexandria,  Virginia:  Jane’s 
Information  Group,  2005),  1,  http://www.janes.com/company/about/  (accessed  10/11/2006). 

74  Ibid,  2. 
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supplied  classified  intelligence  for  a  relatively  homogenous  audience.  The  geographic, 
jurisdictional,  and  disciplinary  diversity  of  NTR  resembles  the  diverse  audience  of 
private  sector  producers  such  as  Jane’s.  This  study  explores  the  impact  of  CBF  used  by 
the  private  sector  on  unclassified  intelligence  distributed  to  NTR.  Given  the  similarly 
between  the  diverse  audiences  of  private  sector  intelligence  producers  and  NTR,  research 
into  other  smart  practices  of  the  private  sector  is  warranted;  the  IC  has  significant 
experience  in  intelligence  analysis,  while  the  private  sector  has  significant  experience  in 
communicating  with  diverse  audiences  with  multiple  interests,  both  of  which  can  be 
exploited  in  improving  intelligence  communication  with  NTR. 

Contextual  background  has  been  established  as  a  critical  factor  in  decision 
making  from  the  time  of  Plato  through  the  current  global  war  on  terror.  The  Department 
of  Homeland  Security  and  Federal  Bureau  of  Investigation  have  established  value  and 
quality  as  two  factors  that  reflect  customer  satisfaction  with  their  unclassified  intelligence 
products,  providing  an  established,  standardized  method  of  measuring  the  impact  of  CBF 
on  those  products.  Technology  acceptance  model  factors  of  usefulness  and  ease  of  use 
also  provide  valuable  feedback  on  the  impact  of  CBF.  This  previously  validated  TAM 
research  provides  an  established,  standardized  method  of  measuring  whether  the 
hyperlinks  utilized  to  achieve  CBF  will  be  accepted  in  widespread  application.  Any 
changes  to  unclassified  intelligence  products  that  are  rated  highly  for  usefulness,  ease  of 
use,  value  and  quality  that  will  be  widely  accepted  while  improving  decision  making  and 
customer  satisfaction  should  be  considered  for  immediate  application. 


32 


III.  HYPOTHESIS  TESTING 


A.  HYPOTHESIS’ 

A  null  hypothesis  (H0)  is  a  hypothesis  that  can  be  statistically  examined.  It  is 
presumed  to  be  true  until  statistical  analysis  demonstrates  it  to  be  false,  or  nullified,  in 
which  case  the  alternative  hypothesis  (Hi)  may  be  accepted.  A  Null  hypothesis  can  be 
designed  to  test  that  there  is  no  difference  between  variables,  it  is  then  evaluated  and  the 
results  examined  to  determine  what  the  probability  is  that  observed  differences  between 
variables  are  by  chance.  In  determining  whether  to  reject  the  null  hypothesis  in  favor  of 
the  alternative  hypothesis,  we  must  detennine  an  appropriate  level  of  significance  that 
must  be  met  to  reject  the  null  hypothesis.  The  smaller  this  p-value  is,  the  more 
significant  the  result;  5%  or  .05  is  generally  accepted  as  significant,  while  1%  or  .01  is 
more  statistically  powerful,  as  it  is  a  much  more  difficult  threshold  to  meet.  In  this 
research  the  following  two  null  hypotheses  are  evaluated  to  determine  whether  to  accept 
the  alternative  hypothesis: 


Ho:  There  is  no  difference  in  intelligence  product  perceived  value  as  represented  by 
DHS/FBI  customer  satisfaction  surveys,  on  average,  with  the  application  of  CBF. 

Hi:  An  intelligence  product  with  hyperlinks  to  open  source  contextual  background  is 
perceived  as  more  valuable  based  on  DHS/FBI  customer  satisfaction  surveys  than  a  non- 
CBF  product,  on  average. 


Ho:  There  is  no  difference  in  intelligence  product  perceived  quality  as  represented  by 
DHS/FBI  customer  satisfaction  surveys,  on  average,  with  the  application  of  CBF. 

Hp  An  intelligence  product  with  hyperlinks  to  open  source  contextual  background  is 
perceived  as  higher  quality  based  on  DHS/FBI  customer  satisfaction  surveys  than  a  non- 
CBF  product,  on  average. 
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B.  METHODOLOGY 

In  this  empirical  research,  a  national  survey  for  homeland  security  professionals 
was  designed  to  discover  the  impact  of  hyperlink  CBF  on  unclassified  intelligence 
products.  A  control  product  was  presented  and  a  base  measurement  of  DHS/FBI  quality 
and  value  factors  was  established.  The  experimental  CBF  product  was  presented  and  a 
second  measurement  of  DHS/FBI  quality  and  value  factors  was  recorded,  establishing  the 
positive  or  negative  impact  of  CBF  on  the  customer  satisfaction  factors  established  by 
DHS/FBI.  Questions  from  previously  validated  technology  acceptance  model  factors  of 
ease  of  use  and  usefulness  were  then  presented  in  order  to  determine  if  intelligence 
recipients  would  accept  hyperlinks  to  achieve  CBF.  A  final  forced  choice  question  to 
determine  preference  for  control  or  experimental  products  was  presented  to  confirm 
DHS/FBI  factor  results. 

1.  Instrument  Construction 

Two  identical  sample  intelligence  products  were  produced  for  the  survey,  one  text 
file  as  similar  as  practical  to  a  typical  intelligence  product  distributed  to  non  traditional 
recipients  and  a  second  identical  product  fused  with  hyperlinks  to  contextual  background 
(CBF)  found  at  reliable,  government  funded  websites  documented  in  Appendix  A.  The 
material  in  the  sample  report  was  unclassified,  having  been  adapted  from  the  Southern 
Poverty  Law  Center  Intelligence  Project  Eco-Radicalism,  open  source,  online  report, 
allowing  this  research  to  remain  unclassified.75  The  intelligence  sample  product  was 
distributed  to  four  intelligence  subject  matter  experts  from  local,  county,  state,  and 
federal  jurisdictions  and  based  on  their  feedback  the  sample  was  shortened. 

The  survey  was  prepared  for  “Zoomerang”  online  distribution.  The  survey 
presented  a  standard  intelligence  product,  asked  seven  questions,  presented  a  CBF 
intelligence  product,  requesting  a  response  to  the  same  seven  questions  along  with  twelve 
additional  questions  directly  related  to  ease  of  use  and  usefulness  consistent  with  the 
Technology  Acceptance  Model  (TAM).  The  first  seven  survey  questions  after  both 
standard  and  CBF  versions  mirror  the  DHS/FBI  customer  satisfaction  survey  with  the 

75  Southern  Poverty  Law  Center,  Eco-Radicalism,  Feds  Across  Country’  to  Smash  Eco-Terrorists  ’ 
(Montgomery,  Alabama:  Southern  Poverty  Law  Center,  2005),  1, 

http://www.splcenter.org/intel/intelreport/article.jsp7aicH608  (accessed  September  22,  2006). 
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questions  divided  into  perceived  value  and  perceived  quality  factors;  Appendix  D 
contains  a  copy  of  this  two  factor  customer  satisfaction  survey  that  is  currently 
distributed  with  DHS/FBI  unclassified  intelligence  products.  These  questions  provide 
direct  feedback  to  DHS/FBI  on  the  impact  of  CBF  on  the  measures  that  they  seek  from 
their  “customers.”  These  questions  utilized  the  same  five-point  Likert  scale  as  DHS/FBI 
products,  with  Strongly  Disagree  on  one  end  and  Strongly  Agree  on  the  other  end,  along 
with  an  N/A  option.  One  question  from  the  DHS/FBI  Customer  Satisfaction  Survey 
regarding  timeliness  was  not  utilized,  “The  product  was  delivered  within  established 
guidelines,”  as  the  sample  product  was  not  being  delivered  based  on  guidelines.  All  other 
customer  satisfaction  survey  questions  and  Likert  scale  are  exactly  as  found  in  DHS/FBI 
documents. 

Twelve  additional  questions  were  adapted  from  Technology  Acceptance  Model 
research  first  validated  in  by  Davis  in  1989.  TAM  was  developed  to  explain  and  predict 
computer  use.76  In  this  model,  the  two  factors  of  “Usefulness”  and  “Ease  of  Use”  have 
been  demonstrated  to  predict  the  use  of  technology.  The  questions  in  these  factors 
utilized  an  ordinal  Likert  6-point  scale  with  Strongly  Disagree  at  one  extreme  and 
Strongly  Agree  at  the  other  end.  A  final  question  in  the  survey  asked  respondents  to 
directly  identify  whether  a  CBF  or  standard  product  would  be  of  greater  value  to  their 
organization. 

The  pilot  study  was  sent  to  fifteen  subject  matter  experts  (SME)  in  public  health, 
fire,  emergency  management,  and  law  enforcement  for  feedback.  The  control  sample 
intelligence  document  was  designed  consistent  with  the  model  below,  with  the  researcher 
producing,  distributing  and  receiving  feedback  on  the  sample  intelligence  document  in  a 
manner  similar  to  existing  intelligence  production. 


76  User  Acceptance  of  Computer  Technology’:  A  Comparison  of  Two  Theoretical  Models,  982-1003. 
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The  experimental  sample  intelligence  document  with  hyperlinks  was  designed 
consistent  with  the  model  below,  with  the  researcher  producing,  distributing  and 
receiving  feedback. 
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The  hyperlinks  utilized  in  the  sample  survey  product  led  to  contextual  background 
at  government  funded  open  source  Internet  sites  vetted  by  the  researcher  (Appendix  A). 
In  addition,  one  hyperlink  to  a  company  website  specifically  identified  in  the  intelligence 
document  was  included,  http://www.huntingdon.com. 

SME  suggested  a  “header”  and  “legalistic  sounding  disclaimer”  to  make  the 
survey  look  more  realistic,  as  well  as  changing  one  jurisdiction  choice  from  “city”  to 
“local.”  After  making  these  adjustments,  the  survey  was  distributed  to  the  target 

audience,  who  were  requested  to  complete  the  survey  and  forward  the  survey  to  other 
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homeland  security  professionals.  As  this  research  utilizes  customer  satisfaction  survey 
questions  from  the  Department  of  Homeland  Security  and  Federal  Bureau  of 
Investigation,  two  major  producers  of  unclassified  intelligence,  a  bifurcated  sampling 
method  was  utilized.  In  one  frame  invitations  to  participate  in  the  survey  were  emailed  to 
current  and  former  students  of  the  DHS  sponsored  Naval  Postgraduate  School,  Center  for 
Homeland  Defense  and  Security  (CHDS).  These  students,  all  of  which  were  selected  by 
DHS  to  attend  the  school,  include  diverse  homeland  security  disciplines  including,  police, 
fire,  public  health,  military,  EMS,  and  emergency  management.  This  sampling  frame  is 
intended  to  represent  intelligence  recipients  of  interest  to  DHS.  As  every  current  and 
former  student  was  invited  to  participate,  these  students  represent  the  entire  universe  of 
CHDS  students  and  no  further  sampling  was  needed  to  survey  this  population. 

A  second  sampling  frame  consisted  of  former  students  of  the  FBI  National 
Academy  (FBINA).  Similar  to  the  DHS  sponsored  CHDS;  each  student  attending  the 
FBINA  is  sponsored  and  selected  by  one  of  the  fifty-six  FBI  Field  Offices.  Each  student 
attending  the  FBINA  is  a  member  of  law  enforcement,  representing  local,  county,  state 
and  international  jurisdictions.  By  2005,  over  38,000  students  from  220  sessions  had 
attended  FBINA,  so  further  sampling  was  required.  Several  domestic  students  from 
FBINA  session  214  were  selected  to  represent  intelligence  recipients  of  interest  to  the 
FBI;  several  CHDS  law  enforcement  survey  recipients  have  previously  attended  FBINA, 
however  none  had  attended  session  214,  avoiding  an  overlap  of  samples.  A  total  of  172 
email  invitations  to  participate  in  the  survey  were  sent  to  current  and  fonner  homeland 
security  professionals  sponsored  by  the  FBI  or  DHS  to  attend  these  courses.  Snowball 
style  secondary  distribution  by  these  initial  recipients  was  encouraged. 

a.  DHS/FBI  Customer  Satisfaction  Survey  Questions 

Quality: 

1 .  The  Product  was  timely  and  relevant  to  your  mission,  programs,  priorities,  or 
initiatives. 

2.  The  product  was  clear  and  logical  in  the  presentation  of  information  with 
supported  judgments  and  conclusions. 

3.  The  product  is  reliable;  i.e.,  sources  well  documented  and  reputable. 
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Value: 

1 .  The  product  would  contribute  to  satisfying  intelligence  gaps  or  predicating 
cases  or  intelligence  operations,  especially  previously  unknown  areas. 

2.  The  product  would  result  in  a  change  in  investigative  or  intelligence  priorities 
and/or  a  shift  from  unaddressed  to  addressed  work,  or  vice  versa. 

3.  The  product  would  result  in  more  informed  decisions  concerning  investigative 
or  intelligence  initiatives  and/or  resource  allocation. 

4.  The  product  would  identify  new  information  associated  with  pending  matters 
or  offered  insights  into  information  that  could  change  the  working  premise  in  a 
program  or  initiative. 

Figure  1  is  a  screen  shot  of  a  DHS/FBI  customer  satisfaction  survey 
distributed  to  NTR. 


b.  Technology  Acceptance  Model 

Perceived  usefulness: 

1 .  Using  the  intelligence  product  with  hyperlinks  in  my  job  would  enable  me  to 
accomplish  tasks  more  quickly. 

2.  Using  intelligence  products  with  hyperlinks  would  improve  my  job 
performance. 

3.  Using  intelligence  products  with  hyperlinks  would  increase  my  productivity. 

4.  Using  intelligence  products  with  hyperlinks  would  enhance  my  effectiveness 

on  the  job. 

5.  Using  intelligence  products  with  hyperlinks  would  make  it  easier  to  do  my  job 

6.  I  would  find  intelligence  products  with  hyperlinks  useful  in  my  job. 

Perceived  ease  of  use: 

1 .  Learning  to  utilize  intelligence  products  with  hyperlinks  would  be  easy  for  me. 

2.  I  would  find  it  easy  to  use  intelligence  hyperlinks  to  obtain  decision  making 
information. 

3.  My  interaction  with  the  intelligence  product  with  hyperlinks  was  clear  and 
understandable. 

4.  I  found  the  intelligence  hyperlinks  to  be  flexible  to  interact  with. 

5.  It  would  be  easy  for  me  to  become  skillful  at  using  intelligence  products  with 

hyperlinks. 

6.  I  found  the  intelligence  product  with  hyperlinks  easy  to  use. 
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c.  Preference 

1 .  Which  type  of  intelligence  product  would  be  of  greater  value  to  you  and  your 

organization? 

The  “Zoomerang”  online  survey  tool  utilized  is  included  as  Appendix  E. 
The  data  obtained  from  the  online  survey  was  downloaded  into  a  Statistical  Package  for 
the  Social  Sciences  (SPSS)  program  for  descriptive,  bivariate,  multivariate,  correlation, 
regression  and  reliability  analysis. 


C.  LIMITATIONS  OF  RESEARCH 

It  should  be  noted  that  the  original  list  of  survey  recipients,  although 
geographically  diverse,  was  based  on  two  advanced  educational  programs,  which  may 
introduce  bias  in  the  results;  these  professionals  had  attended  the  FBI  National  Academy 
or  the  Naval  Postgraduate  School  Center  for  Homeland  Defense  and  Security.  As  DHS 
and  the  FBI  jointly  distribute  unclassified  intelligence  products  and  the  questions  utilized 
in  the  survey  were  in  part  based  on  their  joint  customer  satisfaction  survey,  it  is  believed 
that  sending  the  survey  to  students  of  a  DHS  sponsored  (Center  for  Homeland  Defense 
and  Security),  and  FBI  sponsored  (FBI  National  Academy)  program  would  represent  a 
reasonable  target  audience  for  DHS/FBI  intelligence  products.  Original  distribution  to 
homeland  security  students  and  alumni  of  the  Naval  Postgraduate  School  Center  for 
Homeland  Defense  and  Security  (NPS  CHDS)  and  FBI  National  Academy  (FBINA) 
graduates  may  not  adequately  represent  the  technological  diversity  of  homeland  security 
professionals,  given  the  online  fonnat  of  NPS  CHDS  and  leadership  positions  of  FBINA 
graduates.  Pilot  study  SME  feedback,  participant  free  form  comments  and  demographic 
results  reflect  stronger  ecological  validity  than  external  validity;  while  the  survey  was 
originally  widely  distributed,  the  anonymous  nature  of  feedback  and  low  frequency  of 
individual  jurisdiction/discipline  responses  demands  caution  interpreting  individual 
discipline  and  jurisdiction  results. 

As  research  surveys  were  completed  through  an  anonymous  online  survey 
instrument,  participants  were  not  able  to  be  contacted  for  response  clarification.  The 
“snowball”  distribution  style,  where  those  who  were  originally  requested  to  participate 
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were  encouraged  to  redistribute  to  their  homeland  security  contacts,  reduces  validity, 
while  increasing  sample  size.  This  distribution  style  can  introduce  bias  as  it  reduces  the 
likelihood  that  the  respondents  represent  an  appropriate  population  sample.  Secondary 
distribution  by  these  recipients,  not  controlled  by  the  researcher,  was  intended  to 
similarly  represent  the  target  audience  of  the  DHS  and  the  FBI;  the  extent  to  which  this 
sample  does  not  represent  the  target  audience  of  these  agencies  represents  potential  bias 
in  the  survey  results.77 

As  the  survey  responses  were  anonymous,  recipients  who  visited  the  site  or  only 
partially  completed  the  survey,  then  came  back  to  complete  the  survey,  registered  as  a 
visit,  a  partial  and  a  complete  response.  Since  one  recipient  may  be  included  in  the 
number  of  times  the  site  was  “visited,”  the  number  of  “partial”  responses  and  the  number 
of  “complete”  responses,  only  completed  survey  data  were  utilized  for  research,  and  the 
resulting  visit/partial/complete  response  rates  are  not  utilized.  285  surveys  were 
completed,  182  partial  responses  were  not  utilized,  and  the  website  was  visited  a  total  of 
648  times.  From  this  data,  the  response  rate  may  have  been  anywhere  from  44%  to  100%. 
Not  knowing  the  true  response  rate  is  a  limitation  in  this  research. 

The  sample  was  derived  from  DHS  and  FBI  sponsored  educational  programs  that 
do  not  generally  include  private  sector  participants;  one  result  is  that  the  two  million 
private  sector  security  professionals  are  under  represented  in  this  research. 

D.  RESULTS 

1.  Demographics 

The  sample  contained  285  responses,  all  of  which  were  usable.  Six  discipline 
choices  were  offered  in  the  survey;  law  enforcement  was  the  largest  single  represented 
discipline  with  35.1%  of  responses.  Forty-nine  “Other”  responses  fell  into  the  following 
generalized  categories: 

United  States  Coast  Guard 

Military 

77  Survey  Sampling  Methods,  Stat  Pac,  1,  http://www.statpac.com/surveys/sampling.htm  (accessed 
September  10,  2006). 
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Homeland  Security 

Intelligence 

Public  Works 

Medical 

Legal 

Private  Sector 


Discipline 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  Other 

49 

17.2 

17.2 

Public  Health 

42 

14.7 

14.7 

31.9 

Law  Enforcement 

100 

35.1 

35.1 

67.0 

Fire 

51 

17.9 

17.9 

84.9 

EMS 

7 

2.5 

2.5 

87.4 

Emergency  Management 

36 

12.6 

12.6 

100.0 

Total 

285 

100.0 

100.0 

Table  1.  Discipline  Demographics 


Six  jurisdictions  were  represented;  the  largest  single  jurisdiction  was  local 
respondents  at  41.1%.  Twenty-four  “Other”  responses  fell  into  the  following  general 
categories: 

District  of  Columbia 

Academia 

Tribal 

Multi  County 
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Jurisdiction 


Frequency 

Percent 

Valid  Percent 

Cumulative 

Percent 

Valid  Other 

24 

8.4 

8.4 

8.4 

Federal 

56 

19.6 

19.6 

28.1 

State 

46 

16.1 

16.1 

44.2 

County 

39 

13.7 

13.7 

57.9 

Local 

117 

41.1 

41.1 

98.9 

Private  Sector 

3 

1.1 

1.1 

100.0 

Total 

285 

100.0 

100.0 

Table  2.  Jurisd 

liction  Demographics. 

The  largest  single  discipline/jurisdiction  respondent  combination  was  local  law 
enforcement,  with  fifty  individual  responses  or  17.5%  of  total  respondents,  followed  by 
local  fire  at  14.4%.  Table  2  documents  the  counts  and  percentages  of  each  combination 
of  jurisdiction  and  discipline. 


2.  Responses  DHS/FBI  Quality 

The  three  questions  comprising  the  DHS/FBI  Quality  component  of  customer 
satisfaction  surveys, 

1 .  The  Product  was  timely  and  relevant  to  your  mission,  programs,  priorities,  or 
initiatives. 

2.  The  product  was  clear  and  logical  in  the  presentation  of  information  with 
supported  judgments  and  conclusions. 

3.  The  product  is  reliable;  i.e.,  sources  well  documented  and  reputable, 
were  compared  for  the  standard  and  CBF  products. 
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Question  1 

The  Product  was  timely  and  relevant  to  your  mission,  programs,  priorities,  or 
initiatives. 


Mean 


Discipline 

Jurisdiction 

Standard 
Timely  and 
Relevant 

CBF 

Timely  and 
Relevant 

Other 

Other 

3.00 

3.50 

Federal 

2.74 

3.05 

State 

4.18 

4.55 

County 

2.00 

4.00 

Local 

3.29 

3.43 

Private  Sector 

3.33 

3.67 

Total 

3.22 

3.58 

Public  Health 

Other 

1.88 

2.00 

Federal 

2.00 

2.00 

State 

3.30 

3.33 

County 

3.60 

3.47 

Local 

3.83 

3.67 

Total 

3.15 

3.10 

Law  Enforcement 

Other 

3.25 

3.75 

Federal 

3.33 

3.52 

State 

4.14 

4.00 

County 

3.83 

3.83 

Local 

3.57 

3.96 

Total 

3.60 

3.85 

Fire 

Other 

3.00 

3.00 

State 

2.00 

2.00 

County 

3.14 

3.71 

Local 

3.63 

4.00 

Total 

3.51 

3.88 

EMS 

County 

4.00 

4.00 

Local 

3.67 

3.67 

Total 

3.71 

3.71 

Emergency  Management 

Other 

3.25 

3.75 

Federal 

3.13 

3.25 

State 

3.38 

3.50 

County 

3.11 

3.22 

Local 

3.86 

4.29 

Total 

3.33 

3.56 

Total 

Other 

2.68 

3.00 

Federal 

3.04 

3.25 

State 

3.77 

3.86 

County 

3.41 

3.54 

Local 

3.61 

3.93 

Private  Sector 

3.33 

3.67 

Total 

3.42 

3.66 

Table  3.  DHS/FBI  “Quality”  question  1  response  demographics. 
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State  law  enforcement,  local  and  county  public  health  responses  reflect  a  belief 
that  the  standard  product  was  more  timely  and  relevant  to  their  mission,  programs, 
priorities,  or  initiatives;  federal  public  health,  county  law  enforcement,  state  fire,  other 
fire,  and  all  EMS  respondents  did  not  have  a  preference  in  which  product  was  more 
timely  and  relevant  to  their  mission,  programs,  priorities  while  all  eighteen  others 
preferred  the  CBF  product.  On  average,  public  health  and  federal  respondents  rated  both 
standard  and  CBF  products  lower  than  other  disciplines  and  jurisdictions.  In  no  case  did 
federal  respondents  rate  the  CBF  product  lower  than  the  standard  product;  the  mean 
improvement  of  federal  respondents  was  5.82%.  As  a  discipline,  public  health  rated  the 
CBF  slightly  lower  than  the  standard  product  (1.59%).  Overall,  the  mean  results  for  all 
disciplines  combined  improved  7%  with  the  application  of  CBF  from  3.42  to  3.66  on  the 
6-point  scale. 
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Question  2 

The  product  was  clear  and  logical  in  the  presentation  of  information  with 
supported  judgments  and  conclusions. 


Mean 


Discipline 

Jurisdiction 

Standard  Clear 
and  Logical 

CBF  Clear  and 
Logical 

Other 

Other 

3.25 

4.40 

Federal 

3.45 

3.81 

State 

3.75 

4.25 

County 

2.00 

5.00 

Local 

3.57 

4.14 

Private  Sector 

3.33 

4.00 

Total 

3.49 

4.06 

Public  Health 

Other 

2.67 

3.44 

Federal 

2.50 

2.50 

State 

3.50 

4.00 

County 

3.53 

3.93 

Local 

3.83 

4.17 

Total 

3.33 

3.80 

Law  Enforcement 

Other 

4.00 

4.25 

Federal 

3.50 

4.13 

State 

3.80 

4.27 

County 

3.33 

4.33 

Local 

3.70 

4.12 

Total 

3.66 

4.16 

Fire 

Other 

3.50 

4.00 

State 

3.00 

3.00 

County 

3.00 

4.00 

Local 

4.02 

4.20 

Total 

3.84 

4.14 

EMS 

County 

4.00 

4.00 

Local 

3.33 

4.17 

Total 

3.43 

4.14 

Emergency  Management 

Other 

3.25 

3.75 

Federal 

3.63 

3.75 

State 

3.38 

3.75 

County 

3.33 

3.56 

Local 

4.00 

4.57 

Total 

3.53 

3.86 

Total 

Other 

3.17 

3.88 

Federal 

3.46 

3.89 

State 

3.63 

4.09 

County 

3.33 

3.95 

Local 

3.81 

4.18 

Private  Sector 

3.33 

4.00 

Total 

3.59 

4.05 

Table  4.  DHS/FBI  “Quality”  question  2  response  demographics. 
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No  identified  group  indicated  that  the  standard  product  was  more  clear  or  logical 
in  the  presentation  of  information  with  supported  judgments  and  conclusions,  while 
county  EMS,  state  fire,  and  federal  public  health  showed  no  preference.  Twenty  four 
other  combinations  of  jurisdiction  and  discipline  believed  the  CBF  product  was  more 
clear  and  logical.  Mean  results  improved  approximately  13%  with  the  application  of 
CBF  from  3.59  to  4.05  on  the  6-point  scale.  Although  never  rating  the  standard  product 
higher  than  the  CBF  product,  public  health  and  federal  respondents  continued  to  rate  both 
products  lower  than  the  average  of  all  disciplines  and  jurisdictions. 
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Question  3 

The  product  is  reliable;  i.e.,  sources  well  documented  and  reputable. 

Mean 


Discipline 

Jurisdiction 

Standard 

Reliable 

CBF  Reliable 

Other 

Other 

3.40 

4.20 

Federal 

3.50 

4.33 

State 

3.82 

4.42 

County 

4.00 

5.00 

Local 

3.00 

4.71 

Private  Sector 

3.67 

4.67 

Total 

3.51 

4.43 

Public  Health 

Other 

2.50 

3.67 

Federal 

2.50 

3.50 

State 

3.11 

4.13 

County 

3.27 

3.87 

Local 

2.83 

4.00 

Total 

2.98 

3.88 

Law  Enforcement 

Other 

3.75 

4.75 

Federal 

3.13 

4.17 

State 

3.33 

4.47 

County 

3.83 

4.33 

Local 

3.39 

4.37 

Total 

3.36 

4.35 

Fire 

Other 

3.00 

3.50 

State 

2.00 

2.00 

County 

2.71 

4.29 

Local 

3.78 

4.23 

Total 

3.57 

4.16 

EMS 

Local 

3.17 

4.17 

Total 

3.17 

4.17 

Emergency  Management 

Other 

2.50 

4.50 

Federal 

3.38 

3.88 

State 

2.75 

4.50 

County 

3.00 

3.44 

Local 

3.43 

4.50 

Total 

3.06 

4.09 

Total 

Other 

2.96 

4.08 

Federal 

3.28 

4.16 

State 

3.27 

4.34 

County 

3.21 

3.95 

Local 

3.47 

4.32 

Private  Sector 

3.67 

4.67 

Total 

3.32 

4.22 

Table  5.  DHS/FBI  “Quality”  question  3  response  demographics. 
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No  identified  group  believed  the  standard  product  was  more  reliable;  i.e.,  sources 
well  documented  and  reputable;  state  fire  showed  no  preference  and  twenty  five  other 
combinations  of  jurisdiction  and  discipline  believed  the  CBF  product  was  more  reliable. 
Mean  results  improved  approximately  27%  with  the  application  of  CBF  from  3.32  to  4.22 
on  the  6-point  scale,  the  greatest  improvement  of  any  variable  reviewed.  Public  health 
and  federal  respondents  once  again  rated  both  products  lower  than  the  overall  average  of 
all  disciplines  and  jurisdictions. 


3.  DHS/FBI  Quality 

Appendix  D  reflects  the  intention  of  DHS/FBI  to  define  and  obtain  feedback  on 
intelligence  product  quality  by  requesting  responses  to  the  above  three  questions  under 
the  heading  of  “quality.”  The  individual  responses  to  the  three  questions  were  combined 
to  form  a  DHS/FBI  Quality  factor;  the  mean  of  the  three  quality  questions  was  calculated 
for  both  standard  and  CBF  responses  by  jurisdiction  and  discipline: 
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Mean 


Discipline 

Jurisdiction 

Standard 

Average 

Quality 

CBF  Average 
Quality 

Other 

Other 

3.0833 

3.9167 

Federal 

3.2833 

3.7500 

State 

3.9091 

4.4444 

County 

2.6667 

4.6667 

Local 

3.2857 

4.0952 

Private  Sector 

3.4444 

4.1111 

Total 

3.4130 

4.0355 

Public  Health 

Other 

2.6296 

3.1852 

Federal 

2.3333 

2.6667 

State 

3.4000 

3.8889 

County 

3.4667 

3.7556 

Local 

3.5000 

3.9444 

Total 

3.2222 

3.6341 

Law  Enforcement 

Other 

3.6667 

4.2500 

Federal 

3.4267 

3.9130 

State 

3.8000 

4.2444 

County 

3.6667 

4.1667 

Local 

3.5867 

4.1600 

Total 

3.5867 

4.1190 

Fire 

Other 

3.1667 

3.5000 

State 

2.3333 

2.3333 

County 

2.9524 

4.0000 

Local 

3.8130 

4.1707 

Total 

3.6405 

4.0850 

EMS 

County 

4.6667 

4.6667 

Local 

3.3889 

4.0000 

Total 

3.5714 

4.0952 

Emergency  Management 

Other 

3.0000 

4.0000 

Federal 

3.3750 

3.6250 

State 

3.1667 

3.9167 

County 

3.1481 

3.4074 

Local 

3.7619 

4.5238 

Total 

3.3056 

3.8519 

Total 

Other 

3.0000 

3.6667 

Federal 

3.3273 

3.7610 

State 

3.5926 

4.1259 

County 

3.3419 

3.8291 

Local 

3.6439 

4.1624 

Private  Sector 

3.4444 

4.1111 

Total 

3.4775 

3.9929 

Table  6.  DHS/FBI  “Quality”  factor  response  demographics. 
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When  the  three  questions  comprising  the  DHS/FBI  quality  factor  are  combined, 
county  EMS  and  state  fire  showed  no  preference,  while  all  twenty-five  other 
combinations  of  jurisdiction  and  discipline  indicated  that  the  CBF  product  was  of  higher 
quality  than  the  standard  product.  County  EMS  rated  both  products  very  high  (4.66), 
while  state  fire  rated  both  products  very  low  (2.33).  Mean  results  improved  14.82%  with 
the  application  of  CBF  from  3.48  to  3.99  on  the  6-point  scale;  overall  public  health  and 
federal  respondents  followed  the  same  CBF  improvement  results  as  all  disciplines  while 
rating  both  standard  and  CBF  products  lower  than  other  disciplines/jurisdictions. 


4.  DHS/FBI  Value 

The  four  questions  comprising  the  DHS/FBI  Quality  factor  of  customer 
satisfaction  surveys, 

1 .  The  product  would  contribute  to  satisfying  intelligence  gaps  or  predicating 
cases  or  intelligence  operations,  especially  previously  unknown  areas. 

2.  The  product  would  result  in  a  change  in  investigative  or  intelligence  priorities 
and/or  a  shift  from  unaddressed  to  addressed  work,  or  vice  versa. 

3.  The  product  would  result  in  more  informed  decisions  concerning  investigative 
or  intelligence  initiatives  and/or  resource  allocation. 

4.  The  product  would  identify  new  information  associated  with  pending  matters 
or  offered  insights  into  information  that  could  change  the  working  premise  in  a 
program  or  initiative. 

were  compared  for  the  standard  and  CBF  products. 
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Question  1 

The  product  would  contribute  to  satisfying  intelligence  gaps  or  predicating  cases  or 
intelligence  operations,  especially  previously  unknown  areas. 


Mean 


Discipline 

Jurisdiction 

Standard 

Satisfying 

Gaps 

CBF  Satisfying 
Gaps 

Other 

Other 

3.20 

4.00 

Federal 

2.95 

3.57 

State 

3.50 

4.00 

County 

4.00 

4.00 

Local 

3.29 

3.83 

Private  Sector 

3.67 

4.33 

Total 

3.23 

3.81 

Public  Health 

Other 

2.50 

3.50 

Federal 

3.00 

3.00 

State 

3.20 

3.38 

County 

3.33 

3.80 

Local 

3.50 

4.17 

Total 

3.15 

3.67 

Law  Enforcement 

Other 

3.25 

4.75 

Federal 

3.21 

3.83 

State 

3.20 

4.13 

County 

3.33 

4.33 

Local 

3.26 

4.00 

Total 

3.24 

4.03 

Fire 

Other 

3.00 

3.50 

State 

3.00 

3.00 

County 

3.00 

4.00 

Local 

3.78 

4.03 

Total 

3.63 

3.98 

EMS 

County 

4.00 

3.00 

Local 

3.17 

3.83 

Total 

3.29 

3.71 

Emergency  Management 

Other 

3.75 

4.25 

Federal 

3.00 

3.38 

State 

3.25 

4.00 

County 

2.78 

3.33 

Local 

3.29 

4.29 

Total 

3.14 

3.78 

Total 

Other 

3.04 

3.96 

Federal 

3.07 

3.64 

State 

3.28 

3.91 

County 

3.18 

3.79 

Local 

3.45 

4.02 

Private  Sector 

3.67 

4.33 

Total 

3.28 

3.89 

Table  7.  DHS/FBI  “Value”  question  1  response  demographics. 


52 


Once  again,  federal  responses  paralleled  those  of  other  disciplines  albeit  with 
lower  ratings;  county  EMS  respondents  preferred  the  standard  product,  state  fire,  federal 
public  health,  and  county  “other”  showed  no  preference,  and  the  twenty-three  other 
jurisdiction/discipline  combinations  indicated  that  the  CBF  product  would  better 
contribute  to  satisfying  intelligence  gaps  or  predicating  cases  or  intelligence  operations, 
especially  previously  unknown  areas.  Mean  results  improved  18.6%  with  the  application 
of  CBF  from  3.28  to  3.89  on  the  6-point  scale. 
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Question  2 

The  product  would  result  in  a  change  in  investigative  or  intelligence  priorities 
and/or  a  shift  from  unaddressed  to  addressed  work,  or  vice  versa. 


Mean 


Discipline 

Jurisdiction 

Standard 

Change 

CBF  Change 

Other 

Other 

3.00 

3.40 

Federal 

2.74 

3.30 

State 

3.00 

4.00 

County 

2.00 

5.00 

Local 

3.17 

3.67 

Private  Sector 

3.67 

4.00 

Total 

2.93 

3.62 

Public  Health 

Other 

2.71 

3.50 

Federal 

3.00 

3.00 

State 

2.25 

3.29 

County 

3.00 

3.67 

Local 

3.50 

3.83 

Total 

2.87 

3.55 

Law  Enforcement 

Other 

3.50 

4.00 

Federal 

3.04 

3.67 

State 

3.27 

3.80 

County 

3.17 

3.83 

Local 

3.18 

3.66 

Total 

3.17 

3.71 

Fire 

Other 

2.00 

2.50 

State 

3.00 

2.00 

County 

2.71 

3.57 

Local 

3.19 

3.73 

Total 

3.06 

3.62 

EMS 

County 

4.00 

3.00 

Local 

3.50 

3.67 

Total 

3.57 

3.57 

Emergency  Management 

Other 

3.25 

4.00 

Federal 

2.75 

2.88 

State 

3.14 

3.71 

County 

3.11 

3.44 

Local 

3.57 

4.14 

Total 

3.14 

3.57 

Total 

Other 

2.95 

3.57 

Federal 

2.89 

3.39 

State 

2.98 

3.71 

County 

3.00 

3.64 

Local 

3.24 

3.72 

Private  Sector 

3.67 

4.00 

Total 

3.08 

3.63 

Table  8.  DHS/FBI  “Value”  question  2  response  demographics. 
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County  EMS  and  state  fire  responses  indicate  that  the  standard  product  would 
result  in  a  change  in  investigative  or  intelligence  priorities  and/or  a  shift  from 
unaddressed  to  addressed  work,  or  vice  versa,  federal  public  health  showed  no 
preference,  while  twenty-four  other  jurisdiction/disciplines  preferred  the  CBF  product. 
Mean  results  improved  almost  18%  with  the  application  of  CBF  from  3.08  to  3.63  on  the 
6-point  scale  with  federal  respondents  continuing  their  tendency  to  rate  all  products  lower 
than  other  disciplines. 
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Question  3 

The  product  would  result  in  more  informed  decisions  concerning  investigative  or 
intelligence  initiatives  and/or  resource  allocation. 


Mean 


Discipline 

Jurisdiction 

Standard 

Informed 

CBF  Informed 

Other 

Other 

3.40 

4.20 

Federal 

3.15 

3.76 

State 

3.42 

4.08 

County 

4.00 

5.00 

Local 

2.86 

3.43 

Private  Sector 

3.67 

4.33 

Total 

3.25 

3.90 

Public  Health 

Other 

3.00 

3.63 

Federal 

3.00 

3.50 

State 

2.67 

3.75 

County 

3.80 

4.07 

Local 

4.00 

4.00 

Total 

3.38 

3.87 

Law  Enforcement 

Other 

3.50 

4.50 

Federal 

3.29 

3.83 

State 

3.47 

4.07 

County 

3.33 

4.17 

Local 

3.42 

4.04 

Total 

3.39 

4.02 

Fire 

Other 

3.00 

2.50 

State 

3.00 

3.00 

County 

3.29 

4.14 

Local 

3.62 

3.95 

Total 

3.53 

3.90 

EMS 

County 

3.00 

3.00 

Local 

3.17 

4.17 

Total 

3.14 

4.00 

Emergency  Management 

Other 

3.75 

4.00 

Federal 

2.88 

3.25 

State 

3.14 

4.13 

County 

3.33 

3.44 

Local 

4.00 

4.57 

Total 

3.37 

3.83 

Total 

Other 

3.30 

3.87 

Federal 

3.17 

3.71 

State 

3.23 

4.00 

County 

3.51 

3.95 

Local 

3.50 

4.01 

Private  Sector 

3.67 

4.33 

Total 

3.38 

3.93 

Table  9.  DHS/FBI  “Value”  question  3  response  demographics. 
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“Other”  fire  indicated  that  the  standard  product  would  result  in  more  informed 
decisions  concerning  investigative  or  intelligence  initiatives  and/or  resource  allocation, 
county  EMS,  state  fire,  and  local  public  health  showed  no  preference  and  twenty-three 
other  combinations  indicated  that  the  CBF  product  was  superior  with  federal  respondents 
continuing  to  rate  both  products  lower  than  other  jurisdictions.  Mean  results  improved 
approximately  16%  with  the  application  of  CBF  from  3.38  to  3.93  on  the  6-point  scale. 
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Question  4 

The  product  would  identify  new  information  associated  with  pending  matters  or 
offered  insights  into  information  that  could  change  the  working  premise  in  a 
program  or  initiative. 


Mean 


Discipline 

Jurisdiction 

Standard  New 
Information 

CBF  New 
Information 

Other 

Other 

3.40 

4.20 

Federal 

3.05 

3.71 

State 

3.75 

4.17 

County 

4.00 

5.00 

Local 

3.33 

3.83 

Private  Sector 

4.00 

4.33 

Total 

3.38 

3.96 

Public  Health 

Other 

2.88 

3.63 

Federal 

3.00 

3.50 

State 

2.89 

3.50 

County 

3.80 

3.80 

Local 

3.50 

4.17 

Total 

3.33 

3.74 

Law  Enforcement 

Other 

3.75 

4.25 

Federal 

3.42 

3.96 

State 

3.40 

4.07 

County 

3.33 

4.00 

Local 

3.34 

3.96 

Total 

3.38 

3.99 

Fire 

Other 

3.00 

2.50 

State 

4.00 

4.00 

County 

3.17 

4.00 

Local 

3.63 

4.03 

Total 

3.55 

3.96 

EMS 

County 

3.00 

4.00 

Local 

3.00 

4.33 

Total 

3.00 

4.29 

Emergency  Management 

Other 

3.75 

3.75 

Federal 

3.00 

3.38 

State 

3.50 

3.88 

County 

3.11 

3.44 

Local 

3.57 

4.29 

Total 

3.33 

3.72 

Total 

Other 

3.30 

3.78 

Federal 

3.20 

3.76 

State 

3.42 

3.95 

County 

3.45 

3.82 

Local 

3.44 

4.03 

Private  Sector 

4.00 

4.33 

Total 

3.39 

3.92 

Table  10.  DHS/FBI  “Value”  question  4  response  demographics. 
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“Other”  fire  indicated  that  the  standard  product  would  best  identify  new 
information  associated  with  pending  matters  or  offered  insights  into  information  that 
could  change  the  working  premise  in  a  program  or  initiative,  “other”  emergency 
management,  state  fire,  and  county  public  health  showed  no  preference,  all  twenty-three 
other  combinations  preferred  the  CBF  product.  Mean  results  improved  over  15%  with 
the  application  of  CBF  from  3.39  to  3.92  on  the  6-point  scale.  Federal  respondents 
continued  to  rate  both  products  lower  than  other  jurisdictions  with  emergency 
management  rating  both  products  lower  than  other  disciplines. 
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DHS/FBI  Value 


The  individual  responses  to  the  three  questions  were  combined  to  form  a 
DHS/FBI  Value  factor;  the  mean  was  calculated  for  both  standard  and  CBF 
responses. 

Mean 


Discipline 

Jurisdiction 

Standard 
Average  Value 

CBF  Average 
Value 

Other 

Other 

3.2500 

3.9500 

Federal 

3.0125 

3.6190 

State 

3.4167 

4.0625 

County 

3.5000 

4.7500 

Local 

3.3571 

3.9286 

Private  Sector 

3.7500 

4.2500 

Total 

3.2448 

3.8673 

Public  Health 

Other 

2.8750 

3.5625 

Federal 

3.0000 

3.2500 

State 

2.8333 

3.3214 

County 

3.4833 

3.8333 

Local 

3.6250 

4.0417 

Total 

3.2125 

3.6842 

Law  Enforcement 

Other 

3.5000 

4.3750 

Federal 

3.2396 

3.8229 

State 

3.3333 

4.0167 

County 

3.2917 

4.0833 

Local 

3.3000 

3.9150 

Total 

3.2980 

3.9369 

Fire 

Other 

2.7500 

2.7500 

State 

3.2500 

3.0000 

County 

3.2500 

4.0714 

Local 

3.6188 

3.9408 

Total 

3.5250 

3.8906 

EMS 

County 

3.5000 

3.2500 

Local 

3.2083 

4.0000 

Total 

3.2500 

3.8929 

Emergency  Management 

Other 

3.6250 

4.0000 

Federal 

2.9063 

3.2188 

State 

3.4375 

4.0000 

County 

3.0833 

3.4167 

Local 

3.6071 

4.3214 

Total 

3.2847 

3.7431 

Total 

Other 

3.1848 

3.7935 

Federal 

3.0972 

3.6364 

State 

3.2722 

3.8895 

County 

3.3205 

3.8269 

Local 

3.4440 

3.9605 

Private  Sector 

3.7500 

4.2500 

Total 

3.3143 

3.8556 

Table  11.  DHS/FBI 

“Value”  factor  response  demographics. 
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Appendix  D  reflects  the  intention  of  DHS/FBI  to  define  and  obtain  feedback  on 
intelligence  product  value  by  requesting  responses  to  the  above  three  questions  under  the 
heading  of  “value.”  When  the  three  questions  are  averaged  into  a  “value”  factor,  county 
EMS  and  state  fire  responses  reflect  a  belief  that  the  standard  product  was  of  more  value, 
“other”  fire  showed  no  preference  and  the  twenty-four  other  combinations  of  jurisdictions 
and  disciplines  indicated  that  the  CBF  product  was  of  greater  value;  public  health  and 
federal  responses  rated  all  products  lower  than  other  disciplines  and  jurisdictions.  Mean 
results  improved  16.33%  with  the  application  of  CBF  from  3.3143  to  3.8556  on  the  6- 
point  scale.  Both  “value”  and  “quality”  factors  were  defined  by  the  DHS/FBI  questions 
found  in  Figure  1  under  those  headings. 

Regardless  of  discipline  or  jurisdiction,  the  mean  of  combined  perceived  value 
and  quality  factors  improved  with  the  application  of  CBF. 
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Mean 


Discipline 

Jurisdiction 

Standard 
Average  Value 

CBF  Average 
Value 

Standard 
Average  Quality 

CBF  Average 
Quality 

Other 

Other 

3.25 

3.95 

3.0833 

3.9167 

Federal 

3.0125 

3.619 

3.2833 

3.75 

State 

3.4167 

4.0625 

3.9091 

4.4444 

County 

3.5 

4.75 

2.6667 

4.6667 

Local 

3.3571 

3.9286 

3.2857 

4.0952 

Private 

Sector 

3.75 

4.25 

3.4444 

4.1111 

Total 

3.2448 

3.8673 

3.413 

4.0355 

Public  Health 

Other 

2.875 

3.5625 

2.6296 

3.1852 

Federal 

3 

3.25 

2.3333 

2.6667 

State 

2.8333 

3.3214 

3.4 

3.8889 

County 

3.4833 

3.8333 

3.4667 

3.7556 

Local 

3.625 

4.0417 

3.5 

3.9444 

Total 

3.2125 

3.6842 

3.2222 

3.6341 

Law  Enforcement 

Other 

3.5 

4.375 

3.6667 

4.25 

Federal 

3.2396 

3.8229 

3.4267 

3.913 

State 

3.3333 

4.0167 

3.8 

4.2444 

County 

3.2917 

4.0833 

3.6667 

4.1667 

Local 

3.3 

3.915 

3.5867 

4.16 

Total 

3.298 

3.9369 

3.5867 

4.119 

Fire 

Other 

2.75 

2.75 

3.1667 

3.5 

State 

3.25 

3 

2.3333 

2.3333 

County 

3.25 

4.0714 

2.9524 

4 

Local 

3.6188 

3.9408 

3.813 

4.1707 

Total 

3.525 

3.8906 

3.6405 

4.085 

EMS 

County 

3.5 

3.25 

4.6667 

4.6667 

Local 

3.2083 

4 

3.3889 

4 

Total 

3.25 

3.8929 

3.5714 

4.0952 

Emergency 

Management 

Other 

3.625 

4 

3 

4 

Federal 

2.9063 

3.2188 

3.375 

3.625 

State 

3.4375 

4 

3.1667 

3.9167 

County 

3.0833 

3.4167 

3.1481 

3.4074 

Local 

3.6071 

4.3214 

3.7619 

4.5238 

Total 

3.2847 

3.7431 

3.3056 

3.8519 

Total 

Other 

3.1848 

3.7935 

3 

3.6667 

Federal 

3.0972 

3.6364 

3.3273 

3.761 

State 

3.2722 

3.8895 

3.5926 

4.1259 

County 

3.3205 

3.8269 

3.3419 

3.8291 

Local 

3.444 

3.9605 

3.6439 

4.1624 

Private 

Sector 

3.75 

4.25 

3.4444 

4.1111 

Total 

3.3143 

3.8556 

3.4775 

3.9929 

Table  12.  Combined  DHS/FBI  “Quality”  and  “Value”  factor  demographics. 
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5.  Analysis  DHS/FBI  Value/Quality/Standard/CBF  Comparisons 

In  choosing  statistical  tests  it  is  important  to  detennine  whether  the  results  are 
dependent  or  independent.  If  the  means  (averages)  are  based  on  the  same  people 
answering  all  questions,  the  means  are  considered  dependent,  as  is  the  case  in  this 
research.  Another  critical  step  is  to  determine  whether  the  data  obtained  is  normally 
distributed  (parametric),  or  non  parametric.  By  obtaining  a  basic  knowledge  of  the 
underlying  distribution  of  a  variable,  we  can  make  predictions  about  the  results  of 
repeated  samples.  Normal  distribution  is  reflected  in  a  “bell  curve.”  A  review  of  the  data 
in  this  research  shows  that  it  is  non  parametric.  It  is  similarly  important  to  determine  if 
the  data  is  nominal,  ordinal,  or  scaled.  Nominal  data  are  labels  such  as  “married”  or 
“single”  that  can  be  counted,  but  not  be  ordered  or  measured.  In  this  research, 
jurisdiction  and  discipline  are  nominal  data.  Ordinal  data  are  values  or  observations  that 
can  be  ranked,  or  have  a  rating  scale  attached.  You  can  count  and  order,  but  not  measure 
ordinal  data.  Scaled  data  is  a  measurement  where  the  distance  between  any  two  units  of 
measurements  is  the  same,  but  the  zero  point  is  arbitrary,  not  attached  to  the  rating  scale. 
The  Likert  scales  used  in  this  research  are  ordinal,  with  positive  “whole  number”  integers 
attached  to  a  rating  scale.  For  dependent,  ordinal  results  as  in  this  research,  the  Wilcoxon 
test  for  paired  data  is  appropriate  to  test  the  null  hypothesis  that  there  is  no  difference  in 
responses  to  standard  and  CBF  products,  in  order  to  detennine  if  it  is  appropriate  to  reject 
the  null  hypothesis  (H„)  in  favor  of  the  alternative  hypothesis  (H,).  The  Wilcoxon  test 
computes  the  difference  in  scores,  ranks  those  differences  in  scores,  and  then  computes 
the  mean  rank  for  all  cases  with  a  negative  difference  and  positive  difference.  The 
number  of  scores  in  each  question  varies  due  to  the  N/A  option  provided  in  the  DHS/FBI 
surveys  that  were  dropped  in  this  computation.  The  Wilcoxon  signed  rank  test  is  similar 
to  the  t  test  of  central  tendency;  however  it  is  more  robust  as  it  does  not  assume  a  normal 
distribution.  Given  the  non  parametric  distribution  of  data  observed  in  this  data,  the 
Wilcoxon  signed  rank  test  is  a  more  powerful  test  than  the  t  test. 


63 


N 

Mean  Rank 

Sum  of  Ranks 

CBF  Timely  and  Relevant  - 
Standard  Timely  and 

Negative  Ranks 

22(a) 

47.32 

1041.00 

Relevant 

Positive  Ranks 

76(b) 

50.13 

3810.00 

Ties 

177(c) 

Total 

275 

CBF  Clear  and  Logical  - 
Standard  Clear  and  Logical 

Negative  Ranks 

23(d) 

48.50 

1115.50 

Positive  Ranks 

109(e) 

70.30 

7662.50 

Ties 

148(f) 

Total 

280 

CBF  Reliable  -  Standard 
Reliable 

Negative  Ranks 

9(9) 

48.50 

436.50 

Positive  Ranks 

147(h) 

80.34 

11809.50 

Ties 

1 18(i) 

Total 

274 

CBF  Satisfying  Gaps  - 
Standard  Satisfying  Gaps 

Negative  Ranks 

220) 

60.32 

1327.00 

Positive  Ranks 

137(k) 

83.16 

11393.00 

Ties 

117(1) 

Total 

276 

CBF  Change  -  Standard 
Change 

Negative  Ranks 

15(m) 

56.30 

844.50 

Positive  Ranks 

122(n) 

70.56 

8608.50 

Ties 

133(o) 

Total 

270 

CBF  Informed  -  Standard 
Informed 

Negative  Ranks 

18(p) 

53.08 

955.50 

Positive  Ranks 

120(g) 

71.96 

8635.50 

Ties 

1 38(r) 

Total 

276 

CBF  New  Information  - 
Standard  New  Information 

Negative  Ranks 

21  (s) 

71.43 

1500.00 

Positive  Ranks 

1 32(t) 

77.89 

10281.00 

Ties 

122(u) 

Total 

275 

a  CBF  Timely  and  Relevant  <  Standard  Timely  and  Relevant 
b  CBF  Timely  and  Relevant  >  Standard  Timely  and  Relevant 
c  CBF  Timely  and  Relevant  =  Standard  Timely  and  Relevant 
d  CBF  Clear  and  Logical  <  Standard  Clear  and  Logical 
e  CBF  Clear  and  Logical  >  Standard  Clear  and  Logical 
f  CBF  Clear  and  Logical  =  Standard  Clear  and  Logical 
g  CBF  Reliable  <  Standard  Reliable 
h  CBF  Reliable  >  Standard  Reliable 
i  CBF  Reliable  =  Standard  Reliable 
j  CBF  Satisfying  Gaps  <  Standard  Satisfying  Gaps 
k  CBF  Satisfying  Gaps  >  Standard  Satisfying  Gaps 
I  CBF  Satisfying  Gaps  =  Standard  Satisfying  Gaps 


m  CBF  Change  <  Standard  Change 
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n  CBF  Change  >  Standard  Change 
o  CBF  Change  =  Standard  Change 
p  CBF  Informed  <  Standard  Informed 
q  CBF  Informed  >  Standard  Informed 
r  CBF  Informed  =  Standard  Informed 
s  CBF  New  Information  <  Standard  New  Information 
t  CBF  New  Information  >  Standard  New  Information 
u  CBF  New  Information  =  Standard  New  Information 

Table  13.  DHS/FBI  Value/Quality/Standard/CBF  comparison  ranks. 


CBF 

Timely 

and 

Relevant  - 
Standard 
Timely 
and 

Relevant 

CBF  Clear 
and 

Logical  - 
Standard 
Clear  and 
Logical 

CBF 

Reliable  - 
Standard 
Reliable 

CBF 

Satisfying 
Gaps  - 
Standard 
Satisfying 
Gaps 

CBF 

Change  - 
Standard 
Change 

CBF 

Informed  - 
Standard 
Informed 

CBF  New 
Information 
-  Standard 
New 

Information 

z 

-5.356(a) 

-7.826(a) 

-10.280(a) 

-9.017(a) 

-8.825(a) 

-8.562(a) 

-8.542(a) 

Asymp.  Sig.  (2- 
tailed) 

.000 

.000 

.000 

.000 

.000 

.000 

.000 

a  Based  on  negative  ranks, 
b  Wilcoxon  Signed  Ranks  Test 


Table  14.  DHS/FBI  Value/Quality/Standard/CBF  Z  scores. 


N 

Mean 

Std.  Deviation 

Minimum 

Maximum 

Standard  Average  Quality 

282 

3.4775 

.92315 

1.00 

5.00 

Standard  Average  Value 

280 

3.3143 

.89596 

1.00 

5.00 

CBF  Average  Quality 

280 

3.9929 

.83527 

1.00 

5.00 

CBF  Average  Value 

277 

3.8556 

.89453 

1.00 

5.00 

Table  15.  DHS/FBI  Value/Quality/Standard/CBF  standard  deviations. 


N 

Mean  Rank 

Sum  of  Ranks 

CBF  Average  Quality  -  Negative  Ranks 

25(a) 

65.08 

1627.00 

Standard  Average  Quality  Positive  Ranks 

168(b) 

101.75 

17094.00 

Ties 

85(c) 

Total 

278 

CBF  Average  Value  -  Negative  Ranks 

34(d) 

52.32 

1779.00 

Standard  Average  Value  Positive  Ranks 

178(e) 

116.85 

20799.00 

Ties 

64(f) 

Total 

276 

a  CBF  Average  Quality  <  Standard  Average  Quality 
b  CBF  Average  Quality  >  Standard  Average  Quality 
c  CBF  Average  Quality  =  Standard  Average  Quality 
d  CBF  Average  Value  <  Standard  Average  Value 
e  CBF  Average  Value  >  Standard  Average  Value 
f  CBF  Average  Value  =  Standard  Average  Value 

Table  16.  DHS/FBI  mean  Value/Quality/Standard/CBF  standard  deviations. 
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CBF  Average 
Quality  - 
Standard 
Average 
Quality 

CBF  Average 
Value  - 
Standard 
Average  Value 

z 

Asymp.  Sig.  (2-tailed) 

-9.984(a) 

.000 

-10.684(a) 

.000 

a  Based  on  negative  ranks, 
b  Wilcoxon  Signed  Ranks  Test 

Table  17.  DHS/FBI  Value/Quality/Standard/CBF  Wilcoxon  signed  ranks  test. 


6.  Conclusions:  Perceived  Value  and  Quality 

The  Wilcoxon  test  conducted  examines  the  hypothesis  that  there  is  no  difference 
in  perceived  quality  and  value  between  standard  and  CBF  products.  This  is  not  the  case; 
the  observed  increased  perception  in  both  value  and  quality  would  occur  fewer  than  once 
every  thousand  times  if  there  really  was  no  difference  in  perceived  quality  or  value 
between  the  standard  and  CBF  product.  The  improvement  in  perceived  quality  and  value 
is  statistically  significant.  While  not  included  in  the  original  hypothesis’,  the  Wilcoxon 
test  on  each  of  the  seven  questions  comprising  the  DHS/FBI  quality  and  value  measures 
would  occur  fewer  than  once  every  thousand  times  if  there  really  were  no  difference 
between  the  standard  and  CBF  products.  The  ranks  chart  shows  that  the  CBF  product 
was  rated  higher  by  an  overwhelming  margin,  from  three  to  sixteen  times  higher  on 
individual  variables  (questions)  and  five  to  six  times  higher  when  combined  into  value 
and  quality  factors. 

Reject  H0:  There  is  no  difference  in  intelligence  product  perceived  value  as 
represented  by  DHS/FBI  customer  satisfaction  surveys,  on  average,  with  the  application 
of  CBF  to  open  sources  in  favor  of  Hi:  An  intelligence  product  with  CBF  to  open  sources 
is  perceived  as  more  valuable  based  on  DHS/FBI  customer  satisfaction  surveys  than  a 
non-  CBF  product,  on  average. 

Reject  Ho;  There  is  no  difference  in  intelligence  product  perceived  quality  as 
represented  by  DHS/FBI  customer  satisfaction  surveys,  on  average,  with  the  application 
of  CBF  to  open  sources  in  favor  of  Hi:  An  intelligence  product  with  CBF  to  open  sources 
is  perceived  as  higher  quality  based  on  DHS/FBI  customer  satisfaction  surveys  than  a 
non-CBF  product,  on  average. 
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7.  Responses:  Technology  Acceptance  Model 
a.  Perceived  Usefulness 

The  six  questions  comprising  the  Technology  Acceptance  Model 
“Perceived  Usefulness”  factor  of  the  survey, 

1 .  Using  the  intelligence  product  with  hyperlinks  in  my  job  would  enable  me  to 
accomplish  tasks  more  quickly. 

2.  Using  intelligence  products  with  hyperlinks  would  improve  my  job  performance. 

3.  Using  intelligence  products  with  hyperlinks  would  increase  my  productivity. 

4.  Using  intelligence  products  with  hyperlinks  would  enhance  my  effectiveness  on 

the  job. 

5.  Using  intelligence  products  with  hyperlinks  would  make  it  easier  to  do  my  job 

6.  I  would  find  intelligence  products  with  hyperlinks  useful  in  my  job. 


were  evaluated  to  detennine  the  perceived  usefulness  of  hyperlink  technology  to  achieve 
CBF  in  unclassified  intelligence.  On  the  6-point  Likert  scale  with  6  representing 
“strongly  agree,”  and  1  “strongly  disagree”  with  the  usefulness  of  hyperlink  technology, 
the  mean  varied  from  4.42  to  4.74,  with  a  mode  of  5,  indicating  the  respondents  belief 


that  hyperlink  technology  was  useful  in  this  application. 


Quickly 

Performance 

Productivity 

Effectiveness 

Easier 

Useful 

N  Valid 

285 

285 

285 

285 

285 

285 

Missing 

0 

0 

0 

0 

0 

0 

Mean 

4.61 

4.47 

4.42 

4.49 

4.60 

4.74 

Median 

5.00 

5.00 

5.00 

5.00 

5.00 

5.00 

Mode 

5 

5 

5 

5 

5 

5 

Std.  Deviation 

1.084 

1.105 

1.162 

1.143 

1.098 

1.082 

Variance 

1.176 

1.222 

1.350 

1.307 

1.206 

1.172 

Skewness 

-.853 

-.617 

-.565 

-.633 

-.754 

-.961 

Std.  Error  of  Skewness 

.144 

.144 

.144 

.144 

.144 

.144 

Kurtosis 

.835 

.336 

-.077 

.181 

.409 

1.030 

Std.  Error  of  Kurtosis 

.288 

.288 

.288 

.288 

.288 

.288 

Table  18.  TAM  perceived  usefulness  variable  descriptive  statistics. 
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When  the  six  components  of  perceived  usefulness  are  combined,  a  mean  of  4.5538  with  a 
mode  of  5  was  found. 


Perceived  Usefulness 


N  Valid 

285 

Missing 

0 

Mean 

4.5538 

Median 

4.8333 

Mode 

5.00 

Std.  Deviation 

1 .02732 

Variance 

1.055 

Table  19.  TAM  perceived  usefulness  factor  descriptive  statistics. 


Variation  between  disciplines  was  found,  with  public  health  respondents 
finding  the  hyperlinks  useful,  but  less  so  than  law  enforcement  and  EMS  respondents 
whose  usefulness  mode  was  5  on  the  6-point  Likert  scale.  Variance  is  more  fully 
explored  in  correlation,  regression  and  multiple  regression  segments  of  this  research. 


Discipline 

Average 

Usefulness 

Other 

4.5272 

Public  Health 

4.0873 

Law  Enforcement 

4.6967 

Fire 

4.5915 

EMS 

4.7381 

Emergency  Management 

4.6481 

Total 

4.5538 

Table  20.  TAM  perceived  usefulness  discipline  responses. 
b.  Perceived  Ease  of  Use 

The  six  questions  comprising  the  Technology  Acceptance  Model 
“Perceived  Ease  of  Use”  factor  of  the  survey, 

1 .  Learning  to  utilize  intelligence  products  with  hyperlinks  would  be  easy  for 
me. 

2.  I  would  find  it  easy  to  use  intelligence  hyperlinks  to  obtain  decision 
making  information. 
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3.  My  interaction  with  the  intelligence  product  with  hyperlinks  was  clear  and 
understandable. 

4.  I  found  the  intelligence  hyperlinks  to  be  flexible  to  interact  with. 

5.  It  would  be  easy  for  me  to  become  skillful  at  using  intelligence  products 
with  hyperlinks. 

6.  I  found  the  intelligence  product  with  hyperlinks  easy  to  use. 


were  evaluated  to  determine  the  perceived  ease  of  use  of  hyperlinks  to  achieve  CBF  in 
unclassified  intelligence.  On  the  6-point  Likert  scale  with  6  representing  “strongly 
agree,”  and  1  “strongly  disagree”  with  the  ease  of  use  of  hyperlink  technology,  The  mean 
varied  from  4.66  to  5.01,  with  a  mode  of  5,  indicating  the  respondents  belief  that 
hyperlink  technology  was  easy  to  use  in  this  application. 


Easy 

Decision  Making 

Clear 

Flexible 

Skillful 

Easy  to  Use 

N  Valid 

285 

285 

285 

285 

285 

285 

Missing 

0 

0 

0 

0 

0 

0 

Mean 

5.01 

4.66 

4.84 

4.68 

4.89 

4.98 

Median 

5.00 

5.00 

5.00 

5.00 

5.00 

5.00 

Mode 

5 

5 

5 

5 

5 

5 

Std.  Deviation 

1.026 

1.135 

1.054 

1.067 

1.060 

1.017 

Variance 

1.053 

1.289 

1.110 

1.139 

1.123 

1.035 

Skewness 

-1.183 

-1.004 

-.921 

-.724 

-1.022 

-1.069 

Std.  Error  of  Skewness 

.144 

.144 

.144 

.144 

.144 

.144 

Kurtosis 

.609 

.860 

.713 

.220 

1.078 

1.289 

Std.  Error  of  Kurtosis 

.288 

.288 

.288 

.288 

.288 

.288 

Table  21.  TAM  perceived  ease  of  use  variable  descriptive  statistics. 


When  the  six  components  of  perceived  ease  of  use  are  combined,  a  mean 
of  4.8439  with  a  mode  of  5  was  found. 
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Perceived  Ease  of  Use 


N  Valid 

285 

Missing 

0 

Mean 

4.8439 

Median 

5.0000 

Mode 

5.00 

Std.  Deviation 

.96145 

Variance 

.924 

Table  22.  TAM  perceived  ease  of  use  factor  descriptive  statistics. 

Similar  to  usefulness,  variation  between  disciplines  was  found  in  ease  of 
use  by  discipline;  public  health  respondents  again  found  the  hyperlinks  useful,  but  less  so 
than  law  enforcement  whose  mean  ease  of  use  response  exceeded  5  on  the  6-point  Likert 
scale  with  a  mode  of  5. 


Discipline 

Average  Ease 
of  Use 

Other 

4.8605 

Public  Health 

4.4762 

Law  Enforcement 

5.0417 

Fire 

4.7778 

EMS 

4.8333 

Emergency  Management 

4.7963 

Total 

4.8439 

Table  23.  TAM  perceived  ease  of  use  discipline  responses. 


8.  Conclusions:  Technology  Acceptance 

Survey  data  that  shows  an  improvement  in  both  usefulness  and  ease  of  use 
indicate  a  broad  acceptance  of  hyperlink  technology  to  achieve  CBF  across  disciplines 
and  jurisdictions,  with  strong  evidence  of  support  in  all  twelve  validated  technology 
acceptance  model  questions.  With  a  mode  of  5  out  of  a  possible  6  on  every  question, 
respondents  showed  overwhelming  acceptance  of  hyperlink  technology  use  in 
unclassified  intelligence  to  achieve  CBF.  It  is  important  to  note  that  the  core  of 
technology  acceptance  in  this  research  is  the  perception  that  hyperlinking  of  contextual 
background  to  unclassified  intelligence  products  is  both  useful  and  easy  to  use  for 
homeland  security  professionals. 
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Discipline 

Average  Ease 
of  Use 

Average 

Usefulness 

Other 

4.8605 

4.5272 

Public  Health 

4.4762 

4.0873 

Law  Enforcement 

5.0417 

4.6967 

Fire 

4.7778 

4.5915 

EMS 

4.8333 

4.7381 

Emergency  Management 

4.7963 

4.6481 

Total 

4.8439 

4.5538 

Table  24.  TAM 

[  discipline  responses. 

Jurisdiction 

Average  Ease 
of  Use 

Average 

Usefulness 

Other 

4.6319 

4.0972 

Federal 

4.8869 

4.5863 

State 

5.0217 

4.7391 

County 

4.4316 

4.2821 

Local 

4.9373 

4.6410 

Private  Sector 

4.7222 

4.8889 

Total 

4.8439 

4.5538 

Table  25.  TAM  jurisdiction  responses. 


All  variables  demonstrated  strong  acceptance,  even  the  lowest  ranked  variable, 
productivity,  received  a  score  of  4.42  out  of  6.  Utilizing  TAM  to  predict  the  actual  use  of 
technology,  the  model  below  diagrams  the  strong  individual  variable  inputs  to 
“usefulness”  and  “ease  of  use”  factors  that  contribute  toward  the  intent  to  use  and 
ultimate  predicted  usage  of  CBF  in  unclassified  intelligence.  Previous  research  has 
documented  that  usefulness  is  the  more  important  predictor  of  technology  use  as  no 
matter  how  easy  a  technology  is  to  use,  it  must  be  useful  if  it  is  to  be  accepted. 
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9.  Standard  vs.  CBF  Forced  Choice  Results 

The  final  Likert  question,  “Which  type  of  intelligence  product  would  be  of  greater 
value  to  you  and  your  organization?”  represented  a  direct  choice  for  subjects  between 
CBF  and  standard  unclassified  intelligence  products.  On  the  Likert  scale  forced  choice 
model,  1  represented  “Non-Hyperlinked,”  and  6  “Hyperlinked.” 


Frequency 

Valid  Percent 

Valid 

Non-Hyperlinked 

1 

.4 

2 

4 

1.4 

3 

11 

3.9 

4 

20 

7.0 

5 

61 

21.5 

Hyperlinked 

187 

65.8 

Total 

284 

100.0 

Missing 

System 

1 

Total 

285 

Table  26.  Standard  vs.  CBF  forced  choice  frequency. 


N 

Valid 

284 

Missing 

1 

Mean 

5.45 

Median 

6.00 

Mode 

6.00 

Std.  Deviation 

.929 

Table  27.  Standard  vs.  CBF  forced  choice  descriptive  statistics. 


Valid  responses  from  284  participants  were  obtained  for  this  question  with  187  or 
65.8%  of  respondents  indicating  the  strongest  possible  preference  for  the  hyperlinked 
option  offered,  producing  both  a  median  and  mode  of  6  with  a  mean  of  5.45;  this 
indicates  that  given  a  choice,  homeland  security  professionals  overwhelmingly  preferred 
the  CBF  product.  One  response  was  missing  as  the  survey  did  not  require  a  response  to 
this  question. 

Although  variation  between  disciplines  is  to  be  expected,  all  disciplines  and 
jurisdictions  uniformly  preferred  the  CBF  product  with  means  ranging  from  5.1  to  5.67 


73 


on  the  6-point  Likert  scale  with  law  enforcement,  fire,  local  and  private  sector 
respondents  showing  the  strongest  preference  for  the  CBF  product. 


Table  28. 


Discipline 

Greater  Value 

Other 

5.43 

Public  Health 

5.10 

Law  Enforcement 

5.60 

Fire 

5.59 

EMS 

5.43 

Emergency  Management 

5.33 

Total 

5.45 

Standard  vs.  CBF  forced  choice  discip 


ine  means. 


Jurisdiction 

Greater  Value 

Other 

5.38 

Federal 

5.49 

State 

5.41 

County 

5.13 

Local 

5.57 

Private  Sector 

5.67 

Total 

5.45 

Table  29.  Standard  vs.  CBF  forced  choice  jurisdiction  means. 


10.  Survey  Comments 

Survey  participants  were  offered  an  opportunity  to  make  any  comments  as  a  last 
question  in  the  survey;  109  participants  took  the  opportunity  to  make  comments.  The 
comments  reflected  the  Likert  survey  results,  overwhelmingly  positive,  with  some 
concerns.  Consistent  with  this  research,  many  of  the  concerns  highlighted  the  need  to 
insure  that  all  open  source  contextual  background  used  be  accurate  and  reliable.  Table  3 
includes  all  free  form  comments. 


11.  Correlation  Analysis 

The  Technology  Acceptance  Model  (TAM)  data  in  this  research  is  ordinal  (rank 
order)  and  can  be  ranked  into  two  ordered  series.  The  Spearman  R  test  of  correlation  is 
an  appropriate  test  that  can  be  computed  from  ranks.  In  reviewing  correlations,  all  of 

which  were  significant  at  the  .01  level,  it  is  clear  that  the  forced  choice  question,  “Which 
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type  of  intelligence  product  would  be  of  greater  value  to  you  and  your  organization?”  has 
greater  correlations  to  all  the  technology  acceptance  model  questions  than  DHS/FBI 
quality  and  value  questions.  The  correlation  coefficient  can  range  in  value  from  -1.00  to 
+  1.00,  with  +1.00  indicating  a  perfect,  positive  linear  relationship  and  -1.00  a  perfect, 
inverse  relationship.  Correlations  between  two  variables  does  not  imply  causality,  it  is 
possible  that  other  variables  are  causing  the  correlation  in  whole  or  in  part.  The 
Technology  Acceptance  Model  questions  have  correlation  coefficients  ranging  from 
+.428  to  +.581  with  the  forced  choice  “greater  value”  question,  while  DHS/FBI  value  and 
quality  correlation  coefficients  ranged  from  +.199  to  +.339.  As  the  Department  of 
Homeland  Security  and  FBI  are  interested  in  improving  the  perceived  value  of 
unclassified  intelligence  product  for  individuals  and  organizations,  these  results  warrant 
further  examination.  If  the  forced  choice  “greater  value”  question  results  are  accurate, 
there  is  more  correlation  between  “greater  value”  and  the  validated  TAM  questions  than 
the  DHS/FBI  quality  and  value  questions  currently  in  use. 

Technology  Acceptance  Model  questions  demonstrated  strong  intra  factor 
correlations  between  “Usefulness”  questions  with  r  values  ranging  from  .724-. 880, 
consistent  with  prior  validating  research  on  this  model  and  contributing  to  a  conclusion 
that  the  minor  adaptations  of  original  TAM  questions  necessary  for  this  research 
maintained  that  validity.  TAM  “Ease  of  Use”  showed  similarly  strong  intra  factor 
correlations  ranging  between  .705-. 878  supporting  prior  research.  Discriminate  validity 
is  evidenced  by  the  low  number  of  item  correlations  (2/30)  that  were  higher  between 
“Usefulness”  and  “Ease  of  Use”  than  within  “Usefulness”  and  “Ease  of  Use.”  These 
observations  lead  to  the  conclusion  that  the  questions  represent  two  separate  factors  and 
that  the  questions  discriminate  between  these  two  factors. 

This  conclusion  is  supported  through  a  single  summary  statistic,  Cronbach’s 
Alpha  analysis  of  reliability: 


Cronbach's 

Alpha 

N  of  Items 

.965 

6 

Table  30.  Cronbach’s  Alpha  “Usefulness”  reliability  statistics. 
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A  Cronbach’s  Alpha  analysis  of  reliability  can  be  utilized  to  determine  to  what 
degree  a  survey  maker  was  successful  in  constructing  questions  that  measure  a  person’s 
opinion.  It  provides  a  lower  bound  for  the  true  reliability  of  a  survey,  analyzing  whether 
the  survey  will  differ  because  respondents  have  different  opinions,  as  opposed  to 
confusion  or  multiple  interpretations  of  survey  questions.  The  statistic  is  determined 
based  on  the  number  of  items  in  a  survey,  and  measurements  of  covariance  ratios. 


Cronbach's 

Alpha 

Cronbach's 
Alpha  Based 
on 

Standardized 

Items 

N  of  Items 

.957 

.958 

6 

Table  3 1 .  Cronbach’s  Alpha  “Ease  of  Use”  reliability  statistics. 


Cronbach's 

Alpha 

Cronbach's 
Alpha  Based 
on 

Standardized 

Items 

N  of  Items 

.818 

.824 

3 

Table  32.  Cronbach’s  Alpha  “Quality”  reliability  statistics. 


Cronbach's 

Alpha 

Cronbach's 
Alpha  Based 
on 

Standardized 

Items 

N  of  Items 

.925 

.925 

4 

Table  33.  Cronbach’s  Alpha  “Value”  reliability  statistics. 


Strong  intra  factor  reliability  is  indicated  in  the  Cronbach’s  Alpha  of  each  of  the 
four  factors  utilized  in  this  survey  with  ranges  from  .818  to  .965. 

The  three  questions  representing  the  DHS/FBI  “Quality”  factor  showed  much 
lower  intra  factor  correlations  than  TAM  factors,  ranging  between  .489  and  .679.  The 
“Quality”  factor  questions  were  almost  equally  correlated  with  DHS/FBI  “Value” 
questions  with  correlations  ranging  between  .448-. 645,  a  possible  indication  that  the 
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DHS/FBI  quality  and  value  factor  questions  are  not  measuring  two  separate  factors  of 
value  and  quality;  this  warrants  further  research. 

The  four  questions  within  the  DHS/FBI  “Value”  factor  had  correlations  between 
.685-. 767,  showing  stronger  correlations  than  the  DHS/FBI  “Quality”  measurements,  and 
indicating  that  the  “value”  questions  were  more  closely  measuring  the  same  factor. 
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All  Correlations  are  significant  at  the  0.01  level  (1-tailed). 

Table  34.  Pearson  one  tailed  correlations. 


TAM  Usefulness  Factor 

vl  Quickly 

v2  Performance 

v3  Productivity 

v4  Effectiveness 

v5  Easier 

v6  Useful 
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TAM  Ease  of  Use  Factor 

v7  Easy 

v8  Decision  Making 

-  Cleaf 
vlO  Flexible 
jSkillfli 

vl2  Easy  to  Use 

v  1 3  Greater  V  alue 

DHS/FBI  Quality  Factor 

vl4  Hyperlinked  Timely  and  Relevant 
vl5  Hyperlinked  Clear  and  Logical 
vl6  Hyperlinked  Reliable 

DHS/FBI  Value  Factor 

vl7  Hyperlinked  Satisfying  Gaps 
v  1 8  Hyperlinked  Change 

v  1 9  Hyperlinked  Informed 

v20  Hyperlinked  New  Information 
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12.  Regression  Analysis 

Regression  analysis  investigates  the  relationships  between  variables.  It  can  be 
used  to  seek  the  causal  effect  of  one  variable  upon  another  and  estimates  the  statistical 
significance  of  or  degree  of  confidence  in,  relationships.  In  looking  at  the  impact  of  one 
variable  upon  another,  we  are  conducting  “simple  regression  analysis.”  “Multiple 
regression”  estimates  the  effect  of  each  on  a  dependent  variable  and  helps  explain  the 
impact  of  multiple  simultaneous  variables  on  another  single  variable.  R"  is  the  statistic 
that  measures  the  amount  of  total  variation  that  is  explained  by  the  variable. 

In  this  research,  multiple  regression  analysis  with  the  dependent  variable  of 
“Which  type  of  intelligence  product  would  be  of  greater  value  to  you  and  your 
organization?”  was  conducted  with  technology  acceptance  factors  of  “Usefulness”  and 
“Ease  of  Use,”  along  with  the  DHS/FBI  customer  satisfaction  factors  of  “quality”  and 
“value”  in  order  to  estimate  the  causal  effect  of  the  different  variables  within  each  of 
these  factors  on  increasing  the  value  of  intelligence  products. 

In  determining  the  type  of  multiple  regression  analysis  that  is  appropriate,  it  is 
important  to  screen  the  data  to  determine  if  it  is  linear  in  nature.  The  TAM  “Usefulness” 
graph  with  independent  variables  and  “greater  value”  dependent  value  demonstrates  that 
this  data  is  linear,  indicating  that  a  linear  multiple  regression  is  appropriate.  The 
observed  and  expected  cumulative  probabilities  can  be  observed  to  follow  the  straight 
line  representing  exact  matches  of  observed  and  expected  values. 
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13.  Multiple  Regression  Analysis  Technology  Acceptance  Usefulness 
Factor 


Normal  P-P  Plot  of  Regression  Standardized  Residual 


Dependent  Variable:  greater  value 


0.0  0.2  0.4  0.6  0.8  1.0 


Observed  Cum  Prob 

Greater  value  variable  normal  P-Plot  of  regression  standardized  residual. 


Model 

R 

R  Square 

Adjusted  R 
Square 

Std.  Error  of 
the  Estimate 

1 

.583(a) 

.339 

.325 

.762 

a  Predictors:  (Constant),  Useful,  Quickly,  Productivity,  Easier,  performance, 
Effectiveness 

b  Dependent  Variable:  greater  value 

Table  35.  Multiple  Regression  “Usefulness”  model  summary. 


Model 

Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

1 

Regression 

83.074 

6 

13.846 

23.815 

.000(a) 

Residual 

161.628 

278 

.581 

Total 

244.702 

284 

a  Predictors:  (Constant),  Useful,  Quickly,  Productivity,  Easier,  performance,  Effectiveness 
b  Dependent  Variable:  greater  value 


Table  36.  Multiple  Regression  “Usefulness”  anova. 
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Model 

Unstandardized 

Coefficients 

Standardized 

Coefficients 

t 

Sig. 

B 

Std.  Error 

Beta 

1 

(Constant) 

3.180 

.215 

14.817 

.000 

Quickly 

-.106 

.083 

-.124 

-1.275 

.204 

Performance. 

.030 

.102 

.035 

.290 

.772 

Productivity 

-.096 

.100 

-.120 

-.957 

.339 

Effectiveness 

.254 

.105 

.313 

2.420 

.016 

Easier 

.084 

.098 

.099 

.855 

.393 

Useful 

.323 

.089 

.377 

3.634 

.000 

a  Dependent  Variable:  greater  value 

Table  37.  Multiple  Regression  “Usefulness”  coefficients. 


R  Square  is  an  estimate  of  how  well  the  model  evaluated  explains  the  population 
variance  of  the  dependent  variable,  in  this  case  “Which  type  of  intelligence  product 
would  be  of  greater  value  to  you  and  your  organization?”  In  this  analysis,  all  the 
questions  in  the  TAM  “Usefulness”  factor  together  explain  only  1/3  (.339)  of  the 
dependent  variable,  not  a  substantial  predictor  of  “greater  value.” 

It  is  useful  to  compare  the  “Regression”  sum  of  squares  with  the  “Residual”  sum 
of  squares.  If  the  Regression  is  large  in  relation  to  Residual,  the  model  being  evaluated 
can  be  said  to  account  for  a  large  portion  of  the  dependent  variable  variance.  That  is  not 
the  case  in  “Usefulness,”  the  relatively  high  residual  indicates  that  this  model  fails  to 
explain  much  of  the  variation,  and  calls  for  examination  of  other  variable  sets. 

Another  critical  value  in  this  multiple  regression  is  “t;”  the  t  statistic  indicates  the 
relative  importance  of  each  variable  (question)  in  the  multiple  regression  model  tested. 
Values  substantially  below  -2.00  or  above  2.00  may  be  considered  useful  predictors.  In 
this  analysis,  “Using  intelligence  products  with  hyperlinks  would  enhance  my 
effectiveness,”  and  “I  would  find  intelligence  products  with  hyperlinks  useful  in  my  job” 
are  useful  predictors,  as  both  have  t  above  2  and  are  significant  at  the  .05  and  .01  level 
respectively.  The  other  variables  (questions)  in  the  “Usefulness”  factor  did  not 
substantially  contribute. 

A  multivariate  analysis  utilizing  only  the  two  variables  shown  in  the  full 

“Usefulness”  factor  analysis  to  have  a  t  statistic  over  2  demonstrates  that  these  two 

variables  account  for  almost  all  the  variance  predicted  by  the  six  variables  in 
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“Usefulness.”  The  R  Square  in  this  case  actually  increases  from  .339  to  .377,  indicating 
the  two  questions  alone  better  explain  the  dependent  variable  variance  than  all  six 
questions  together,  although  the  two  question  model  still  fails  to  explain  much  of  the 
variation. 


Model 

R 

R  Square 

Adjusted  R 
Square 

Std.  Error  of 
the  Estimate 

1 

.614(a) 

.377 

.373 

.735 

a  Predictors:  (Constant),  easy  to  use,  decision  making 
b  Dependent  Variable:  greater  value 


Table  38.  “Usefulness”  two  variable  multiple  regression  model  summary. 


Model 

Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

1 

Regression 

92.354 

2 

46.177 

85.475 

.000(a) 

Residual 

152.348 

282 

.540 

Total 

244.702 

284 

a  Predictors:  (Constant),  easy  to  use,  decision  making 
b  Dependent  Variable:  greater  value 


Table  39.  “Usefulness”  two  variable  multiple  anova. 


14.  Multiple  Regression  Analysis  Technology  Acceptance  Ease  of  Use 
Factor 


Model 

R 

R  Square 

Adjusted  R 
Square 

Std.  Error  of 
the  Estimate 

1 

.623(a) 

.388 

.375 

.734 

a  Predictors:  (Constant),  easy  to  use,  decision  making,  Flexible,  Easy,  Skillful,  and 
Clear 

b  Dependent  Variable:  greater  value 

Table  40.  Multiple  Regression  “Ease  of  Use”  model  summary. 


Model 

Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

1 

Regression 

95.036 

6 

15.839 

29.421 

.000(a) 

Residual 

149.666 

278 

.538 

Total 

244.702 

284 

a  Predictors:  (Constant),  easy  to  use,  decision  making,  Flexible,  Easy,  Skillful,  and  Clear 
b  Dependent  Variable:  greater  value 


Table  41.  Multiple  Regression  “Ease  of  Use”  anova. 
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Model 

Unstandardized 

Coefficients 

Standardized 

Coefficients 

t 

Sig. 

B 

Std.  Error 

Beta 

1 

(Constant) 

2.503 

.228 

10.975 

.000 

Easy 

.146 

.093 

.161 

1.569 

.118 

decision  making 

.125 

.060 

.153 

2.097 

.037 

Clear 

-.033 

.104 

-.038 

-.323 

.747 

Flexible 

.095 

.080 

.109 

1.183 

.238 

Skillful 

.026 

.100 

.029 

.254 

.800 

easy  to  use 

.248 

.110 

.271 

2.261 

.025 

a  Dependent  Variable:  greater  value 

Table  42.  Multiple  Regression  “Ease  of  Use”  coefficients. 


Although  slightly  higher  than  “Usefulness,”  “Ease  of  Use”  R  Square  of  .388  does 
not  substantially  predict  the  variance  of  “greater  value.”  The  large  Regression  compared 
to  Residual  will  cause  us  to  continue  to  look  for  variance  sets  that  better  explain  “greater 
value”  variance.  Similar  to  “Usefulness,”  only  two  variables,  “I  would  find  it  easy  to  use 
intelligence  hyperlinks  to  obtain  decision  making  information”  and  “I  found  the 
intelligence  product  with  hyperlinks  easy  to  use”  have  t  statistics  above  2,  making  them 
useful  predictors  of  dependent  variable  variance. 


15.  Multiple  Regression  Analysis  DHS/FBI  Quality  Factor 


Model 

R 

R  Square 

Adjusted  R 
Square 

Std.  Error  of 
the  Estimate 

1 

.375(a) 

.141 

.131 

.876 

a  Predictors:  (Constant),  H  reliable,  H  timely  and  relevant,  H  clear  and  logical 
b  Dependent  Variable:  greater  value 

Table  43.  Multiple  Regression  DHS/FBI  quality  factor  model  summary. 


Model 

Sum  of 
Squares 

df 

Mean  Square 

F 

1 

Regression 

33.687 

3 

1 1 .229 

14.624 

.000(a) 

Residual 

205.783 

268 

.768 

Total 

239.471 

271 

a  Predictors:  (Constant),  H  reliable,  H  timely  and  relevant,  H  clear  and  logical 
b  Dependent  Variable:  greater  value 


Table  44.  Multiple  Regression  DHS/FBI  quality  factor  anova. 
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Model 

Unstandardized 

Coefficients 

Standardized 

Coefficients 

t 

Sig. 

B 

Std.  Error 

Beta 

1 

(Constant) 

3.763 

.269 

14.000 

.000 

H  timely  and  relevant 

-.040 

.064 

-.046 

-.625 

.532 

H  clear  and  logical 

.115 

.089 

.114 

1.286 

.199 

H  reliable 

.327 

.083 

.312 

3.951 

.000 

a  Dependent  Variable:  greater  value 

Table  45.  Multiple  Regression  DHS/FBI  quality  variable  coefficients. 


As  might  be  expected  after  reviewing  the  bivariate  correlation  results  that  showed 
the  individual  correlations  of  the  technology  acceptance  variables  to  have  stronger 
correlations  with  “greater  value”  than  the  DHS/FBI  Quality  and  Value  factors,  the 
multiple  regression  analysis  of  “DHS/FBI  Quality”  has  less  explanatory  power  than  the 
TAM  factors.  The  “Quality”  R  Square  indicates  that  only  14%  of  dependent  variable 
variance  is  explained  by  this  factor,  the  high  Regression  to  Residual  ratio  causes  us  to 
look  for  other  explanations,  and  only  one  of  the  three  questions,  “The  product  is  reliable; 
i.e.,  sources  well  documented  and  reputable”  has  a  t  statistic  above  2,  albeit  the  highest 
yet  at  3.951.  A  review  of  the  “reliable”  variable  alone  shows  only  a  minor  drop  in 
variance  explanation  from  14%  for  all  three  variables  to  13%  with  the  single  variable. 


Model 

R 

R  Square 

Adjusted  R 
Square 

Std.  Error  of 
the  Estimate 

1 

.363(a) 

.132 

.128 

.874 

a  Predictors:  (Constant),  H  reliable 
b  Dependent  Variable:  greater  value 


Table  46.  Multiple  Regression  DHS/FBI  quality  “reliable”  variable  model  summary. 

16.  Multiple  Regression  Analysis  DHS/FBI  Value  Factor 


Model 

R 

R  Square 

Adjusted  R 
Square 

Std.  Error  of 
the  Estimate 

1 

.325(a) 

.106 

.092 

.894 

a  Predictors:  (Constant),  H  new  information,  H  change,  H  satisfying  gaps,  H  informed 
b  Dependent  Variable:  greater  value 

Table  47.  Multiple  Regression  DHS/FBI  value  factor  model  summary. 
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Model 

Unstandardized 

Coefficients 

Standardized 

Coefficients 

t 

Sig. 

B 

Std.  Error 

Beta 

1 

(Constant) 

4.156 

.240 

17.314 

.000 

H  satisfying  gaps 

.061 

.096 

.066 

.640 

.523 

H  change 

.004 

.091 

.004 

.040 

.968 

H  informed 

.048 

.108 

.051 

.448 

.654 

H  new  information 

.217 

.098 

.226 

2.221 

.027 

a  Dependent  Variable:  greater  value 

Table  48.  Multiple  Regression  DHS/FBI  value  factor  variable  coefficients. 


Minimum 

Maximum 

Mean 

Std. 

Deviation 

N 

Predicted  Value 

4.49 

5.81 

5.45 

.305 

273 

Residual 

-4.043 

1.514 

.000 

.887 

273 

Std.  Predicted  Value 

-3.149 

1.181 

.000 

1.000 

273 

Std.  Residual 

-4.523 

1.693 

.000 

.993 

273 

a  Dependent  Variable:  greater  value 


Table  49.  Multiple  Regression  DHS/FBI  value  factor  residual  statistics. 


The  “Value”  factor  showed  the  lowest  explanatory  power  with  an  R  Square  of 
.106,  or  less  than  11%  of  dependent  variable  variance.  Only  one  variable,  “The  product 
would  identify  new  infonnation  associated  with  pending  matters  or  offered  insights  into 
information  that  could  change  the  working  premise  in  a  program  or  initiative”  had  a  t 
statistic  above  2,  and  Predicted/Residual  indicates  a  poor  explanation  of  dependent 
variable  variance 


17.  Multiple  Regression  Analysis  Summary 

The  analysis  of  how  well  each  factor  explains  the  variance  of  “Which  type  of 
intelligence  product  would  be  of  greater  value  to  you  and  your  organization?”  shows  that 
“Ease  of  Use,”  the  most  significant  factor  with  an  R  Square  of  0.388,  explains  less  than 
40%  of  “greater  value”  variance.  The  TAM  factors  of  Usefulness  and  Ease  of  Use 
demonstrated  dramatically  greater  explanatory  power  than  the  DHS/FBI  Quality  and 
Value  factors. 
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Factor 

R  Square 

Usefulness 

0.339 

Ease  of  Use 

0.388 

Quality 

0.141 

Value 

0.106 

Table  50.  Multiple  Regression  Usefulness/Ease  of  Use/Quality/Value  R  square. 


The  following  diagram  summarizes  both  explanatory  R  Square  (R2)  and 
individual  variable  correlations  (R)  with  greater  value.  In  this  diagram  it  is  clear  that  the 
highest  correlated  DHS/FBI  variable,  “The  product  is  reliable;  i.e.,  sources  well 
documented  and  reputable”  equals  the  correlation  of  “usefulness,”  the  lowest  TAM 
variable  at  .339,  supporting  findings  that  overall  technology  acceptance  modeling  is  a 
better  predictor  of  “greater  value”  ratings  than  current  DHS/FBI  questions. 
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Figure  7.  Factor  multiple  regression  and  individual  variable  correlation  coefficient 

relationships  with  “greater  value”  variable. 


Conducting  a  multiple  regression  analysis  with  the  individual  t  statistics  that  were 
over  2  for  each  factor  yielded  the  largest  R  square  at  .416,  indicating  that  this  set  of 
variables  accounts  for  approximately  41%  of  “greater  value”  variance.  “I  found  the 
intelligence  product  with  hyperlinks  easy  to  use”  has  the  only  substantial  t  statistic  in  this 
analysis;  at  5.223,  it  still  explains  less  dependent  variable  variance  than  unknown 
variables  as  indicated  by  the  constant  t  statistic  of  8.33 1 
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Model 

R 

Adjusted  R 
Square 

Std.  Error  of 
the  Estimate 

1 

.645(a) 

.416 

.403 

.725 

a  Predictors:  (Constant),  Hyperlinked  New  Information,  Easy  to  Use,  Hyperlinked 
Reliable,  Effectiveness,  Decision  Making,  Useful 
Table  51.  t  statistic  >  2  multiple  regression  model  summary. 


Model 

Sum  of 
Squares 

df 

Mean  Square 

F 

Sig. 

1 

Regression 

99.629 

6 

16.605 

31.559 

.000(a) 

Residual 

139.954 

266 

.526 

Total 

239.582 

272 

a  Predictors:  (Constant),  Hyperlinked  New  Information,  Easy  to  Use,  Hyperlinked  Reliable, 
Effectiveness,  Decision  Making,  Useful 
b  Dependent  Variable:  Greater  Value 


Table  52.  t  statistic  >  2  multiple  regression  anova. 


Unstandardized 

Coefficients 

Standardized 

Coefficients 

Model 

B 

Std.  Error 

Beta 

t 

Sig. 

1 

(Constant) 

2.213 

.266 

8.331 

.000 

Useful 

.076 

.084 

.086 

.901 

.369 

Effectiveness 

.108 

.071 

.129 

1.523 

.129 

Decision  Making 

.073 

.065 

.086 

1.117 

.265 

Easy  to  Use 

.369 

.071 

.387 

5.223 

.000 

Hyperlinked  Reliable 

.023 

.063 

.022 

.370 

.711 

Hyperlinked  New 
Information 

.027 

.058 

.028 

.466 

.642 

a  Dependent  Variable:  Greater  Value 

Table  53.  t  statistic  >  2  coefficients. 


Reviewing  “Easy  to  Use”  alone  shows  an  R  Square  of  .355,  that  largest  single 
explanatory  variable;  continued  review  of  Regression/Residual  shows  a  high  Residual 
and  the  t  statistic  shows  that  this  variable  is  almost  equal  to  the  constant. 


Model 

R 

R  Square 

Adjusted  R 
Square 

Std.  Error  of 
the  Estimate 

1 

.596(a) 

.355 

.353 

.747 

a  Predictors:  (Constant),  Easy  to  Use 


Table  54.  “Easy  to  Use”  model  summary. 


89 


Model 

Sum  of 
Squares 

df 

Mean  Square 

F 

1 

Regression 

86.826 

1 

86.826 

155.640 

.000(a) 

Residual 

157.876 

283 

.558 

Total 

244.702 

284 

a  Predictors:  (Constant),  Easy  to  Use 
b  Dependent  Variable:  Greater  Value 


Table  55.  “Easy  to  Use”  anova. 


Model 

Unstandardized 

Coefficients 

Standardized 

Coefficients 

T 

Sig. 

B 

Std.  Error 

Beta 

1 

(Constant) 

2.750 

.221 

12.420 

.000 

Easy  to  Use 

.544 

.044 

.596 

12.476 

.000 

a  Dependent  Variable:  Greater  Value 

Table  56.  “Easy  to  Else”  coefficients. 


The  multiple  variable  regression  analysis  of  the  variables  in  this  research  does  not 
substantially  explain  responses  to  “Which  type  of  intelligence  product  would  be  of 
greater  value  to  you  and  your  organization?”  The  multicollinearity  of  the  results 
indicates  that  there  may  be  variables  outside  of  those  examined  in  this  research  that  better 
explain  “greater  value”  variance.  Further  research  to  produce  validated  DHS/FBI 
customer  satisfaction  survey  questions  is  warranted. 


E.  CONCLUSIONS 

Previous  theory  of  reasoned  action  and  technology  acceptance  model  research  has 
provided  empirical  evidence  of  the  relationship  between  TAM  ease  of  use  and  usefulness 
factors  and  actual  system  use.  The  demonstrated  validity  of  Technology  Acceptance 
Model  questions  regarding  “Usefulness”  and  “Ease  of  Use”,  combined  with 
overwhelming  acceptance  of  the  hyperlink  technology  in  this  research  strongly  predicts 
the  acceptance  of  hyperlink  technology  to  achieve  CBF  in  unclassified  intelligence  across 
the  broad  range  of  homeland  security  disciplines  and  jurisdictions,  extending  the 
application  of  TAM  to  homeland  security  settings.  In  this  research,  homeland  security 
professionals  have  demonstrated  a  clear  intention  to  use  hyperlinks  to  achieve  CBF, 
strongly  predicting  actual  use. 
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This  predicted  acceptance,  paired  with  definitive  feedback  that  the  CBF  product 
would  be  of  greater  value  to  organizations  and  the  statistically  significant  improvements 
in  DHS/FBI  “Value”  and  “Quality”  factors  indicates  that  this  concept  has  the  potential  to 
significantly  improve  the  provision  of  unclassified  intelligence  to  millions  of  homeland 
security  professionals.  CBF  has  been  demonstrated  to  be  one  tool  that  can  assist  in 
meeting  the  demands  and  recommendations  of  the  Markle  Foundation,  NTR,  the  9/11 
Commission,  California  Police  Chiefs,  DHS,  Major  City  Police  Chiefs,  the  CIA,  the 
National  Strategy  for  Homeland  Security,  the  National  Infrastructure  Advisory  Council, 
and  the  RAND  Corporation  cited  in  this  research.  CBF  of  unclassified  intelligence  for 
NTR  contributes  toward: 

•  “...news. .  .in  context.”  (9/1 1  Commission,  cited  on  page  1  of  this  research) 

•  “...adequate  context  for  homeland  security  providers  to  effectively  utilize 
information... specific,  tailored  for  each  local  entity,  rapidly  disseminated,  and 
does  not  overburden  recipients  with  vague  or  irrelevant  information.”  (Markle 
Foundation,  page  1) 

•  Improves  law  enforcement  perception  of  intelligence  quality  and  value. 
(California  Police  Chiefs,  page  2) 

•  Is  a  “better  method  of  sharing  intelligence  with  state  and  local  law  enforcement 
agencies?”  (DHS,  page  2) 

•  Contributes  toward  providing  “adequate... background... for  700,000  officers  on 
the  street.”  (Major  City  Police  Chiefs,  page  6) 

•  “...harness(es)  the  power  of  information  technology...”  (Markle  Foundation, 
page  8) 

•  Provides  “background  and  contextual  information  available  at... policymaker’s 
fingertips  in  a  timely  fashion...”  (CIA,  pagel  1) 

•  Moves  toward  “full  integration  of  (the)  private  sector  into  the  intelligence  cycle.” 
(National  Infrastructure  Advisory  Council,  page  16) 

•  Encourages  “effective  partnership  with  state  and  local  government  and  the  private 
sector...”  (National  Strategy  for  Homeland  Security,  page  15) 

•  Is  “tailored  to  meet  requests  from  government  departments,  police,  and  private 
industry.”  (RAND,  page  25) 
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Research  has  found  that  even  well  educated  decision  makers  who  understand  the 
power  of  statistical  testing  often  rely  on  analogies  or  metaphors  to  change  deeply  held 
beliefs,  analogical  reasoning  combined  with  statistical  analysis  is  more  effective  in 
changing  those  beliefs  than  statistics  alone.78 

In  describing  the  power  of  context  in  terrorism  decision  making  Professor  Fathali 
Moghaddam  contends  that  “..terrorism  is  explained  by  the  power  of  context...”  and 
provides  the  following  analogy: 

...although  John  could  choose  to  wear  a  bright  red  and  yellow  tie  at  his 
uncle’s  funeral  in  New  York,  he  is  very  likely  to  choose  a  dark,  somber  tie 
instead;  just  as  when  John  gets  married,  his  future  wife  could  choose  to 
follow  an  alternative  fashion  trend  and  wear  a  black  dress  at  their 
wedding,  but  she  is  far  more  likely  to  wear  a  white  dress.  In  each  case,  the 
individual  choices,  John  deciding  to  wear  a  dark  tie  at  a  funeral  and  his 
bride  deciding  to  wear  a  white  dress  at  their  wedding,  are  made  with  some 
measure  of  free  will  and  we  can  hold  the  individuals  responsible  for  their 
choices,  but  we  must  also  accept  that  their  choices  were  heavily  influenced 
by  the  social  contexts  (of  a  funeral  and  a  wedding).79 

Another  contextual  metaphor/urban  legend  about  the  FBI  describes  how  J.  Edgar 
Hoover  demanded  short  memos  with  wide  margins  (in  order  to  make  notations),  then 
received  a  memo  with  narrow  margins.  He  promptly  returned  the  memo  with  the 
notation,  “Watch  the  borders.”  Subordinates  promptly  sent  hundreds  of  agents  to  guard 
our  national  borders  with  Canada  and  Mexico! 

Whether  it  is  as  mundane  as  dressing  appropriately  for  a  funeral,  as  expensive  as 
sending  hundreds  of  agents  to  the  borders,  or  as  critical  as  meeting  the  unclassified 
intelligence  needs  of  millions  of  homeland  security  professionals,  context  matters.  This 
research  demonstrates  that  through  CBF  it  is  practical  for  the  Intelligence  Community  to 
provide  NTR  the  context  required  to  make  good  decisions.  Context  that  is  consistent 
with  the  henneneutic  of  the  originating  agency,  is  immediately  available  to  decision 
makers,  is  “fused”  to  keywords,  is  available  to  over  87,000  jurisdictions,  improves  both 

quality  and  value  of  intelligence  products  and  utilizes  technology  that  is  widely  accepted. 

78  Nagy,  Thomas,  Theresa  Jefferson  and  Jamal  Altorkistani,  Selling  Proven  Methods  for  Improving 
the  Usability  of  the  Web:  Experimental  Evidence  of  the  Benefits  of  Metaphors  for  Supporting  Heuristic 
Evaluation,  http://home.gwu.edu/~nagy/Hfweb99.htm.  (  accessed  October  18,  2006),  1. 

79  Moghaddam,  Fathali,  From  the  Terrorists '  Point  of  View  What  They  Experience  and  Why  They 
Come  to  Destroy  (Westport,  London:  Praeger  Security  International,  2006),  1 1 
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IV.  RECOMMENDATIONS 


Research  has  outlined  the  critical  impact  of  context  on  decision  making,  the 
importance  of  intelligence  in  homeland  security,  the  need  to  involve  millions  of  diverse 
homeland  security  professionals  in  asymmetric  conflict,  and  the  importance  of 
technology  in  intelligence  operations.  Statistical  analysis  of  the  data  indicates  that 
hyperlinking  unclassified  intelligence  products  to  open  source  contextual  background 
(CBF)  increases  the  perceived  value  and  quality  of  that  intelligence.  Given  a  choice, 
homeland  security  professionals  overwhelmingly  preferred  a  CBF  product,  268  to  16. 
Observed  increases  in  perceived  value  and  quality  and  clear  preference  for  CBF  in 
intelligence  products  make  it  clear  that  CBF  positively  impacts  information  sharing  as 
demanded  by  national  directives,  initiatives  and  homeland  security  professionals.  A 
critical  evaluation  of  these  findings  utilizing  technology  acceptance  model  research 
indicates  that  not  only  does  CBF  improve  unclassified  intelligence  products;  the  use  of 
hyperlink  technology  to  achieve  CBF  is  widely  accepted  by  NTR.  Strong  ease  of  use  and 
usefulness  findings  across  disciplines  and  jurisdictions  predict  that  NTR  will  use  this 
technology  if  employed  in  unclassified  intelligence  production  and  distribution.  In  order 
to  improve  information  sharing  to  and  between  NTR  as  demanded,  the  following 
recommendations  should  be  considered: 

•  Unclassified  intelligence  producers  with  the  capability  should  immediately 
fuse  vetted,  accurate,  open  source  contextual  background  to  their  intelligence 
products  through  the  use  of  hyperlink  technology. 

Homeland  security  professionals  have  expressed  acceptance  of  hyperlink 
technology  in  unclassified  intelligence  distribution.  Intelligence  producers  such  as 
NYPD  have  successfully  integrated  other  technologies  such  as  PowerPoint  into 
unclassified  intelligence  production  and  advanced  contextual  background  technology 
such  as  “Intelink,”  “K2,”  “Profiler,”  and  “Autonomy”  are  successfully  used  in  classified 
intelligence  systems.  This  Department  of  Homeland  Security  sponsored  research  has 
demonstrated  the  acceptance  of  technology  across  a  broad  range  of  homeland  security 
disciplines  and  the  effectiveness  of  TAM  in  evaluating  technology  prior  to  widespread 
homeland  security  application. 
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•  DHS  should  immediately  sponsor  research  to  detennine  what  other 
technologies,  if  any,  would  be  acceptable  and  improve  perceived  value  and 
quality  in  unclassified  intelligence  products.  More  sophisticated  contextual 
background  systems  such  as  K2,  Profiler  or  Autonomy,  community  of  interest 
search  systems  prioritizing  results  based  on  the  search  history  of  homeland 
security  professionals  in  similar  disciplines/jurisdictions/geographic  areas, 
discipline  specific  products,  products  with  discipline  specific  contextual 
background  hyperlinks  and  PowerPoint  are  all  technologies  that  are  utilized 
by  some  jurisdictions  or  in  applications  outside  unclassified  intelligence  that 
should  be  evaluated  utilizing  TAM  for  broad  homeland  security  application. 
Technology  that  is  found  useful  and  easy  to  use  should  be  considered  for 
widespread  application  to  improve  information  sharing. 


This  research  has  demonstrated  that  statistically  significant  improvements  in 
DHS/FBI  defined  perceived  value  and  quality  factors  along  with  strong  TAM  ease  of  use 
and  usefulness  factor  ratings  are  possible  with  the  addition  of  open  source  contextual 
background  to  unclassified  intelligence.  NTR  found  the  simple  addition  of  contextual 
background  to  unclassified  intelligence  improved  that  intelligence,  contributing  to 
improved  information  sharing  as  demanded  by  national  directives,  initiatives  and 
homeland  security  professionals. 

•  The  Department  of  Homeland  Security  should  sponsor  research  into  what 
other  information,  data  or  intelligence  components  would  improve  perceived 
quality  and  value  of  unclassified  intelligence  products  for  NTR.  The  preferred 
length  of  unclassified  intelligence,  the  optimal  amount  of  contextual 
background,  the  inclusion  of  sources  for  further  information,  citations,  and  the 
fusion  of  related  online  training  to  unclassified  intelligence  products  are 
examples  of  changes  to  unclassified  intelligence  that  should  be  evaluated 
utilizing  TAM. 


Interpol  acts  as  a  distributor  of  unclassified  intelligence  products  produced  by  all 
member  agencies  to  other  member  nations  as  opposed  to  being  a  producer  of  unclassified 
intelligence  themselves  and  regional  organizations  in  the  U.S.  have  successfully 
distributed  unclassified  intelligence  produced  by  different  disciplines  and  jurisdictions 
(such  as  the  California  OES  distribution  of  NYPD  unclassified  intelligence  Power 
Points). 
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•  The  Department  of  Homeland  Security  should  investigate  the  feasibility  of 
distributing  the  unclassified  intelligence  products  of  allied  agencies  such  as 
NYPD  in  distribution  systems  under  their  control  a  la  Interpol. 


The  regional  fusion  center  concept  has  demonstrated  an  ability  to  coordinate 
multiple  disciplines  in  counter  terrorism  efforts. 

•  The  Department  of  Homeland  Security  should  continue  to  support  the  fusion 
center  concept,  encouraging  the  use  of  these  centers  to  “tailor”  intelligence  to 
the  unique  needs  of  their  multidisciplinary  partners  and  geographic  area  of 
responsibility,  consistent  with  study  recommendations.  Additionally,  DHS 
should  sponsor  research  into  additional  methods  of  geographic  “tailoring”  of 
unclassified  intelligence. 


Analysis  of  responses  to  DHS/FBI  customer  satisfaction  survey  questions 
produced  results  that  reflect  low  correlation  with  “greater  value;”  there  appears  to  be 
multicollinearity  with  multiple  questions  measuring  the  same  phenomena  along  with 
some  individual  questions  that  do  not  substantially  contribute  to  ‘value”  and  “quality” 
factors  with  potential  overlap  between  questions  and  factors,  while  previously  validated 
TAM  questions  regarding  “usefulness”  and  “ease  of  use”  continued  to  demonstrate 
statistical  validity. 

•  DHS/FBI  should  continue  to  collect  data  on  customer  satisfaction,  partnering 
with  the  Naval  Postgraduate  School,  Center  for  Homeland  Defense  and 
Security,  providing  ongoing  data  to  the  university  and  working  collaboratively 
to  analyze  data,  refine  customer  satisfaction  (intelligence  feedback)  survey 
questions,  and  leverage  the  academic  rigor  and  diverse  knowledge,  skills  and 
abilities  of  staff  and  students  to  improve  unclassified  intelligence. 


The  Naval  Postgraduate  School,  Center  for  Homeland  Defense  and  Security  has 
established  itself  as  a  successful  educational  model,  bringing  well  educated,  experienced 
and  diverse  professionals  together  in  a  collaborative,  focused  effort  to  improve  homeland 
defense  and  security.  Every  discipline  and  jurisdiction  represented  in  the  cohorts  has  a 
demonstrated  need  for  unclassified  intelligence,  and  this  research  reflects  a  desire  and 
ability  to  receive  improved  unclassified  products.  The  need  for  quality  unclassified 

intelligence  is  a  significant  commonality  across  the  diverse  disciplines,  jurisdictions,  and 
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positions  of  CHDS  students;  the  vast  majority  of  employees  in  every  parent  agency  do 
not  have  security  clearances,  potentially  making  unclassified  intelligence  needs  the  most 
significant  common  denominator  at  CHDS.  It  is  practical  to  leverage  the  knowledge, 
skills  and  abilities  of  CHDS  students  along  with  the  sponsorship  of  DHS  to  improve 
unclassified  intelligence. 

•  The  Naval  Postgraduate  School,  Center  for  Homeland  Defense  and  Security 
should  initiate  a  specific  course  on  unclassified  intelligence,  utilizing  the 
collective  knowledge,  skills  and  abilities  of  cohort  members  to  obtain 
feedback  and  produce  original  research  on  network  centric  strategies  to 
improve  unclassified  intelligence  production  and  distribution  for  respective 
employing  agencies. 

As  the  Center  for  Homeland  Defense  and  Security  approaches  the  graduation  of 
200  Homeland  Security  Professionals,  an  opportunity  to  leverage  this  critical  mass  of 
field  tested,  academically  qualified  and  geographically  dispersed  research  partners  exists 
for  DHS.  Each  participant  of  CHDS,  individually  selected  by  DHS  to  represent 
jurisdictional,  disciplinary,  and  geographic  diversity  within  the  practice  of  U.S.  homeland 
security  has  demonstrated  a  significant  commitment  toward  ongoing  improvements  in 
homeland  security.  More  than  a  university  based  educational  program,  CHDS’  unique 
access  to  these  diverse  professionals,  contacts  within  multiple  levels  of  government  and 
developing  university  partnerships  provides  an  opportunity  to  take  the  next  step, 
combining  these  assets  to  further  assist  homeland  security  through  increased  applied 
academic  research.  Unclassified  intelligence  research  provides  an  opportunity  to 
significantly  impact  homeland  security  through  the  millions  of  NTR  without  security 
clearances  who  make  the  bulk  of  public  contacts  every  day.  While  this  “Wal-Mart” 
homeland  security  intelligence  for  everyone,  everywhere,  may  lack  the  public  appeal  of 
top  secret,  need  to  know  intelligence  research,  it  addresses  the  needs  of  those  who 
actually  get  the  work  done  on  a  day  to  day  basis,  the  patrol  officers  responsible  for  80% 
of  all  arrests,  firefighters,  emergency  medical  technicians,  and  public  health  professionals 
responding  daily  to  emergency  calls  for  service,  and  the  two  million  security 
professionals  protecting  85%  our  nation’s  infrastructure. 
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•  CHDS  should  immediately  initiate  an  unclassified  intelligence  research  grant 
in  partnership  with  DHS,  utilizing  current  and  former  students  and  faculty  to 
integrate  the  recommendations  contained  in  this  research,  identifying  specific 
actions  that  support  the  national  vision  and  demands  of  NTR. 
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APPENDIX  A.  CONTEXTUAL  BACKGROUND  WEBSITES 


http://newark.fbi.gOv/dojpressrel/2006/nk030206usa.htm 
http://www.tkb. org/Category.jsp?catID=6 

http://www.mipt.org/pdf/Military-Guide-to-Terrorism-Twenty-First-Century.pdf 

http://www.nal.usda.gov/awic/legislat/pll02346.htm 

http://www.tkb.org/KeyLeader.jsp?memID=61 14 

http  ://www .  tkb .  org  /  Group  .j  sp?  groupID=42 3 4 

http  ://www.  fbi.  gov/wanted/  alert/ sandiego_da.htm 

http  ://www.  tkb  .org/ Group  .j  sp?groupID=  1 4 

http  ://www.  tkb  .org/ Group  .j  sp?groupID=4 1 

http://www.fbi.gov/publications/terror/terror99.pdf 
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APPENDIX  B.  COUNTS  AND  PERCENTAGES 


%  within 
discipline 


%  within 
jurisdiction 


%  of  Total 


Jurisdiction 

Other 

Federal 

State 

County 

Local 

Private 

Sector 

5 

21 

12 

1 

7 

3 

10.2% 

42.9% 

24.5% 

2.0% 

14.3% 

6.1% 

20.8% 

37.5% 

26.1% 

2.6% 

6.0% 

100.0% 

1 .8% 

7.4% 

4.2% 

.4% 

2.5% 

1.1% 

9 

2 

10 

15 

6 

0 

21.4% 

4.8% 

23.8% 

35.7% 

14.3% 

.0% 

37.5% 

3.6% 

21.7% 

38.5% 

5.1% 

.0% 

3.2% 

.7% 

3.5% 

5.3% 

2.1% 

.0% 

4.0% 

25.0% 

15.0% 

6.0% 

50.0% 

.0% 

16.7% 

44.6% 

32.6% 

15.4% 

42.7% 

.0% 

1 .4% 

8.8% 

5.3% 

2.1% 

17.5% 

.0% 

2 

0 

1 

7 

41 

0 

3.9% 

.0% 

2.0% 

13.7% 

80.4% 

.0% 

8.3% 

.0% 

2.2% 

17.9% 

35.0% 

.0% 

.7% 

.0% 

.4% 

2.5% 

14.4% 

.0% 

0 

0 

0 

1 

6 

0 

.0% 

.0% 

.0% 

14.3% 

85.7% 

.0% 

.0% 

.0% 

.0% 

2.6% 

5.1% 

.0% 

.0% 

.0% 

.0% 

.4% 

2.1% 

.0% 

49 


100.0% 

17.2% 


17.2% 


42 


100.0% 


14.7% 


14.7% 


100 


100.0% 


35.1% 


35.1% 


51 


100.0% 


11.1% 

22.2% 

22.2% 

25.0% 

19.4% 

.0% 

16.7% 

14.3% 

17.4% 

23.1% 

6.0% 

.0% 

1 .4% 

2.8% 

2.8% 

3.2% 

2.5% 

.0% 

24 

56 

46 

39 

117 

3 

8.4% 

19.6% 

16.1% 

13.7% 

41.1% 

1.1% 

100.0% 

100.0% 

100.0% 

100.0% 

100.0% 

100.0% 

8.4% 

19.6% 

16.1% 

13.7% 

41.1% 

1.1% 

100.0% 


2.5% 


2.5% 


36 


100.0% 


12.6% 


12.6% 

285 


100.0% 


100.0% 


100.0% 
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APPENDIX  C.  COMMENTS 


30.  Comments? 

# 

Response 

1 

It  was  not  clear  early  on  that  we  were  to  compare  the  difference 
between  info  with,  and  without  hyperlinks.  That  should  be  pointed  out 
early. 

2 

1  think  you  should  add  a  sub-category  to  distinguish  LE  intel  from  other 

LE  groups.  The  info  in  the  bulletin  isn't  going  to  change  anything  most 
intel  units  are  going  to  do  because  they  should  already  have  info,  and  it 
won't  be  considered  timely.  Now,  non-intel  might  give  another 
response. 

3 

The  hyper  linked  document  allows  me  to  get  more  information  in  the 
areas  1  need. 

4 

Easy  to  understand  and  source,  especially  the  hyperlinked  version. 
Excellent  example  of  quality  intelligence  dissemination  at  an 
unclassified  level. 

5 

None 

6 

The  format  with  hyperlinks  is  by  far  the  better  intelligence  product. 

7 

Hyperlinks  would  not  necessarily  make  the  job  faster  because  it  could 
take  longer  for  the  investigator/employee/administrator  to  navigate  the 
hyperlinks  however  the  hyperlinks  would  be  invaluable  to  those  who 
would  choose  to  use  them. 

8 

Information  with  contact  or  source  information  is  most  relevant  to 
ensure  quality  reporting/actions 

9 

Though  the  two  documents  said  basically  the  same  thing  the  second 
one  was  much  easier  to  follow  because  1  could  look  up  what  1  didn't 
know  and  determine  its  worth  to  my  organization. 

10 

In  my  current  day-to-day  job  1  wouldn't  react  to  this  type  of  info  without 
further  authority  to  do  so  but,  if  1  was  directed  to  react,  the  hyperlinks 
would  be  very  useful  since  there  wasn't  enough  information  in  the  non- 
hyperlinked  info  for  me  to  have  a  good  grasp  of  the  situation. 

11 

A  hyperlink  with  the  classified  ID  of  the  1 1  and  their  MOs. 
a  list  of  potential  secondary  sites, 
a  capability  to  share  with  "significant  private  partners" 
a  financial  cost/risk  column  for  cross  reference  and  potential 
escalation/de-escalation  analysis. ..(this  allows  for  speculation  to  future 

103 


hardening  or  MOs) 

nice  job. ..this  can  be  translated  into  other  sub  disciplines  i.e. ,  EMS 
public  health  and  fire  intel  sharing  you  may  want  o  speak  to  Jim  Tindall 
502  or  I  can  send  you  my  thoughts  about  creating  a  national  medical 
intel  center  which  could  greatly  benefit  from  this  concept 


12 

Hyperlinks  can  also  be  used  in  a  similar  manner  as  explanatory 
footnotes  in  a  research  paper.  Hyperlinks  provided  further  explanation  if 
the  reviewer  was  not  familiar  with  the  subject,  group,  tactic,  etc. 
referenced  in  the  intelligence  report. 

13 

Hyperlinks  benefit  in  a  military  environment  might  be  of  limited  value 
unless  a  cross-domain  security  solution  exists.  Much  intel  analysis  is 
done  on  JWICS  and  SIPRNET.  These  systems  do  not  have  much 
interconnection  with  unclass  networks. 

14 

1  know  it  is  just  a  sample  but  both  missed  important  decision  data.  What 
region  of  the  state  (if  available  or  say  it  is  not).  Descriptors  for  wanted 
subjects  (photos  if  available)  and  the  names  of  known  group  members. 

15 

1  feel  the  hyper  links  add  an  added  component  to  the  Intel  product 
which  can  be  used  at  the  customer  needs. 

16 

The  use  of  hyperlinks  is  a  great  idea.  The  initial  bulletin  is  like  an 
executive  summary.  The  hyperlinks  provide  the  opportunity  for  to  gain 
more  information  if  needed. 

17 

The  hyperlinks  provided  access  to  the  sources  of  information  that  were 
helpful  in  developing  a  complete  picture  with  background. 

18 

The  hyperlinks  made  the  report  more  credible  and  interesting  than  the 
example  without  supporting  documentation.  The  only  suggestion  1 
would  make  is  to  limit  the  graphics  if  possible  to  speed  up  download 
time.  The  last  hyperlink  (FBI  1999  Terrorism  Report)  took  too  long  to 
download  and  this  may  cause  some  people  to  be  frustrated. 

19 

The  problem  with  hyperlinks  is  that  they  need  to  be  kept  active.  1 
clicked  on  one  which  might  not  be  active  on  purpose  but  then  1  would 
have  to  spend  the  time  to  research  the  link  if  it  was  information  that  1 
thought  was  necessary  to  read  or  maybe  1  felt  well  if  the  links  aren't 
active  then  maybe  the  report  isn't  credible  and  1  shouldn't  bother  with  it. 

20 

The  questionnaire  asked  for  my  discipline,  and  then  it  removed  N/A  as 
an  answer.  Most  of  the  questions  regarding  intelligence  and  job 
effectiveness  should  have  been  answered  with  N/A  from  an  emergency 
management  perspective,  but  that  answer  was  not  available. 

21 

This  looks  like  a  great  project!  I'll  be  curious  to  see  the  final  proposal 
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and  recommendations-especially  considerations  regarding 
classification/distribution,  etc.  This  might  not  be  part  of  the  scope  of  the 
project,  but  it  would  be  interesting  to  have  a  strategic  planning  group 
come  up  with  a  fact  sheet  with  the  information  too--the  links  are  great, 
but  time  constraints  may  delay  reading  important  information. 

Excellent  product.  I  can't  tell  you  how  excited  I  would  be  to  have  a 
report  such  as  to  work  off  of. 

22 

Please  push  this  out  as  soon  as  you  can.  With  limited  time  and 
resources  these  days,  we  need  all  the  help  we  can  get. 

I  use  hyperlinks  within  documents  and  pdf's  in  most  of  my  work.  If  the 
hyperlink  does  not  exist  in  the  PDF  then  I  place  hyperlinks  within  the 

23  document.  Hyperlinks  are  very  important  in  locating  important 
supporting  information.  However,  the  hyperlinks  MUST  be  link  to  the 
proper  reference  otherwise  the  link  would  be  misleading. 

The  addition  of  hyperlinks  was  an  improvement.  The  number  of 

24  hyperlinks  used,  however  was  distracting.  Not  all  of  them  worked,  and 
took  more  time  to  look  at  than  it  did  to  read  the  article. 

Public  Health  in  this  region  is  VERY  rarely  included  in  the  intel  loop. 

25  This  is  very  different  from  my  former  location  where  the  hyperl inked 
version  would  have  been  very  useful. 

When  using  a  date  in  the  summary,  I  would  include  the  year  to  avoid 

26  confusion  and  allow  the  reader  to  quickly  see  the  age  of  the  data  before 
deciding  to  view  the  ink. 

The  use  of  the  hyperlinked  information  is  a  real  time  saver  and  is  the 
obvious  choice  for  the  busy  manager.  My  only  concern  is  there  may  be 
a  tendency  for  the  manager  to  become  lazy  and  rely  only  on  the 
information  presented  without  independent  research  on  his/her  part. 

The  selection  of  the  most  correct  and  possibly  diverse  sources  of 
hyperlinked  information  is  critical  to  his/her  success. 

27 

From  the  Emergency  Management  perspective,  the  intelligence 
information  will  allow  us  to  prepare  for  those  events  that  may  not  be 
prevented  even  though  we  have  some  intelligence  indicating  they  are  a 
possibility  in  the  near  future.  This  is  valuable  information  for  the 
emergency  manger  assuming  he/she  is  cleared  to  inform  those  that 
need  the  information  to  prepare. 

Great  idea  -  logical,  not  sure  if  current  SOGs  for  Intel  products  already 

28  include  this.  Would  be  interesting  to  know.  Some  of  what  I  get  -  FOUO 
stuff  -  does  use  hyperlinks. 
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29  What  a  great  simple  change. 

30  Thank  you... 

The  fire  service  doesn't  get  or  use  much  in  the  way  of  advanced 
intelligence  for  resource  allocation.  I  have  strained  my  brain  trying  to 
find  an  intelligence  "niche"  for  the  FS,  but  it  seems  to  be  very  small.  For 
large  departments  (such  as  mine),  if  the  advanced  warning  time  is 

31  sufficient,  we  could  use  it  for  specialized  training  in  the  potential  target 
facility,  but  developing  this  takes  time.  Adding  a  list  of  potential  targets-- 
or  past  target  structure  types--(e.g.,  testing  laboratories,  Federal 
buildings,  etc.)  for  the  group  involved  would  be  helpful. ..maybe  as  a 
hyperlink? 

Great  idea  to  be  able  to  drill  down  to  obtain  additional  information  on 

32  hyperlinked  items.  How  about  a  color  coded  system,  to  be  able  to 
discern  between  Names,  Organizations,  Attacks. 

The  second  intelligence  document  with  the  hyperlinks  provided  much 
more  information,  documented  credible  sources  and  also  allowed  for 
the  logical  flow  of  the  report.  It  was  much  better  than  the  first  report 
without  hyperlinks.  The  portion  of  the  intelligence  product  that  was 
missing  was  the  SO  WHAT,  an  Assessment  from  the 
information/intelligence  presented  in  the  report  and  recommendation  of 
actions,  if  any  for  the  user  of  the  product  to  take  as  a  result  of  the 
intelligence  report. 

Hyperlinks  are  a  great  resource  tool  if  you  follow  a  few  simple 
guidelines:  1 .  Hyperlinks  must  be  accurately  setup  on  a  key  word  rather 
than  an  entire  sentence.  2.  Hyperlinks  must  point  directly  to  the 
relevant  source.  (News  Media  and  many  websites  move  links  around 
constantly  so  what  is  available  during  the  report  may  not  be  available 
after  distribution.  3.  Keep  the  number  of  links  down  to  the  most 
important  topics,  and  a  brief  overview  of  the  source  is  still  needed. 
(Follow  an  MLA  format  and  there  shouldn't  be  any  problems.) 

Hyperlinks  better  than  no  links.  Kicker  is  that  the  hyperlinks  are  only 
additive  if  the  information  at  the  linked  site  is  useful.  It's  either  GI/GO  or 
the  'other  guy'  has  an  informative,  useful  page.  Using  the  examples 
given,  predominantly  a  civilian  law  enforcement  example,  most  of  the 
information  available  is  already  provided.  Would  it  truly  be  beneficial  to 

35  know  when  Mr.  San  Diego  was  arrested  for  car  theft  when  he  was  15? 

I  must  say  this  is  a  good  effort.  My  view  is  from  DoD,  and  my  personal 
view  is  that  you  kill  bad  guys,  you  don't  arrest  them.  This  survey  is  very 
domestic  focused,  and  seems  to  reinforce  the  view  that  9/1 1  et  al,  are 
law  enforcement  issues.  This  misses  the  point  that  such  events  are 
perpetrated  by  foreign,  non-GENEVA  signatories;  they  are  hostile  and 
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lethal,  and  researching  them  like  a  criminal  case  will,  in  the  long  run, 
hurt  the  Nation. 

I  don't  know  if  there  is  a  better  answer  within  the  US  Homeland.  Too 
many  lawyers  and  too  much  bureaucracy. 

I  found  the  intelligence  report  with  the  hyperlinks  provided  me  with 
more  information  in  about  the  same  amount  of  time  as  the  report 
without  the  links.  I  did  not  see  additional  information  that  would  have 

36  changed  the  priorities  of  my  job.  I  think  one  of  the  values  I  could  see 
from  the  hyperlinks  is  the  ability  to  build  a  "living"  intelligence  report 
with  additional  information  added  as  it  becomes  available,  resulting  in 
more  timely  and  specific  information. 

This  was  a  very  useful  exercise.  I  do  believe  that  the  way  information  is 
imparted  to  analysts  allows  them  to  collate  more  effectively;  however,  it 
is  still  the  quality  of  the  information,  rather  than  just  the  way  it  is 
presented,  which  is  most  important  to  the  EM  aspects  of  intelligence 
operations.  For  instance,  since  EM  is  a  response/planning  discipline, 
any  operational  aspect  has  to  be  proactive;  so,  in  order  for  additional 
value  to  be  added,  EOC's  have  to  be  activated,  teams  assembled, 

37  information  disseminated. ..and  yet,  if  the  information  is  valueless  or 
without  credibility,  it  makes  it  all  that  much  more  difficult  to  convince  the 
decision  makers  that  a  decision  needs  to  be  made.  The  hyperlinks 
were  valuable,  but  I  would  venture,  only  from  a  research  viewpoint. 
Second-flight  incarnations  of  this  platform  could  have  small  "notes"  in 
place  of  the  hyperlinks  so  that  one  doesn't  even  have  to  wait  for  the 
hyperlinks  to  activate;  as  the  mouse  is  moved  over  the  hyperlink,  the 
notes  immediately  pop  up.  This  would  certainly  make  the  information 
gathering  and  situational  awareness  process  more  timely. 

38  Loved  the  use  of  hyperlinks  -  more  efficient  and  time-saving  for  me! 

Since  I  work  in  "health"  we  have  more  limited  access  to  the  background 
information  that  traditional  LE  has  access  to.  Therefore,  the  ability  to 
3Q  get  additional  information  via  a  hyperlink  provides  good  background 
information  for  review  and  assistance  in  considering  the  potential  health 
impacts  or  need  specific  to  changes  in  "medical/public  health" 
intelligence  gathering  such  as  syndromic  surveillance  etc. 

40  Hyper  links  embedded  in  documents  is  a  great  idea 

The  concern  with  hypertext  links  is  the  tracking  of  web  links  by  the 
source.  While  useful,  the  hypertext  links  really  would  just  help  novices 
close  the  initial  learning  gap.  For  example,  SHAC  is  well  known  for 
those  of  us  who  monitor  the  AE  activities  throughout  the  nation. 

Absent  secure  links  I  would  be  hesitant  to  visit  some  hypertext  linked 


107 


sites,  especially  the  unknown  websites. 


Special  thanks  for  conducting  research  in  this  important  area  of  HSHD. 


42 

1  do  not  use  the  type  of  info  reported  on  in  product  for  my  job,  but  1  like 
the  tool  itself,  and  found  it  very  easy  to  use 

43 

In  my  present  position  the  two  scenarios  would  not  benefit  me.  1 
answered  the  questions  based  on  me  being  employed  in  a  position  that 
could  use  this  type  of  intelligence. 

44 

Those  of  us  in  Environmental  Health  who  will  participate  in  responding 
in  the  event  of  a  disaster  or  Security  incident  need  more  information... of 
threats... not  just  notification  of  an  event.  1  wonder  about  the  security  of 
the  notification  system. ..how  will  we  prevent  it  from  getting  into  the 
wrong  hands? 

45 

You  change  your  scoring  halfway  thru  the  last  survey.  You  start  with  6 
meaning  N/A,  and  then  it  switches  to  strongly  agree.  Some  may  miss 
this  transition.  It's  obvious  you're  testing  the  use  and  viability  of 
hyperlinks,  not  sure  you  need  to  ask  so  many  questions  to  get  there. 
Good  survey  and  good  luck. 

46 

The  resources  are  easier  to  use  with  hyperlinks  and  are  more  likely  to 
be  used.  A  good  start  in  general.  Need  more  useable  information  in  the 
field.  Hyperlinks  would  be  a  good  idea  on  all  levels,  classified  or  non- 
classified. 

47 

The  hyperlinked  report  allowed  for  greater  authentication,  increased  my 
view  as  to  credibility  and  allowed  me  to  "see"  the  supporting  detail  and 
concerns  for  myself. 

48 

one  of  the  links  could  not  be  found  on  the  Justice  Department  server 

49 

The  hyperlink  allowed  me  to  access  information  that  interested  me,  and 
quickly  get  the  "meat"  of  the  intel  report  vs.  having  to  weed  through  a 
lot  of  "stuff".  The  hyperlink  provided  an  opportunity  to  deeper  into  the 
topics  that  interested  me,  thus  allowing  me  to  focus  my  reading  time 
more  efficiently. 

50 

Great  concept  -  keep  it  up. 

51 

This  is  a  great  foundation. ..1  would  like  to  see  this  with  an  analytical 
product  that  identifies  trends  and  threats  -  and  to  be  completely  selfish 
that  effect  the  fire  service,  hazmat,  EMS  and  Urban  Search  and 

Rescue. 

52 

The  hyperlinked  example  was  familiar  to  me  primarily  because  of  my 
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previous  use  of  "wikipedia"  -  it  was  easy  to  use  and  provided  more 
information.  1  thought  the  second  example  was  definitely  a  better 
product,  easier  to  use,  and  provided  opportunity  for  greater  knowledge. 

53 

1  like  the  versatility  of  the  hyperlink,  if  1  didn’t  know  the  subject  well  1 
could  click  the  hyperlink  and  get  the  background 

54 

excellent  use  of  a  common  tool 

55 

Great  idea  -  potentially  very  useful.  My  only  concern  was  info  overload  - 
while  it  should  improve  productivity  1  would  have  a  tendency  to  delve 
too  deeply  into  the  supporting  information  and  might  lose  sight  of  the 
focus  of  the  intel  product. 

56 

1  found  the  Hyper  Links  to  be  an  all  around  better  format. 

57 

Hyper  links  offer  the  reader  the  opportunity  to  locate  more  detailed 
information  or  additional  sources  of  information.  Great  idea! 

58 

Hyperlinks  are  useful,  but  could  be  more  information  than  needed.  We 
operate  in  an  information  overload  and  do  not  need  excess,  non  useful 
information.  Good  Intel  does  not  necessarily  mean  more  Intel  but  it 
needs  to  be  relative  to  the  product  I'm  using. 

59 

There  is  an  obvious  difference  in  the  quality  and  depth  of  the  two 
products.  Unfortunately,  1  have  seen  way  too  many  of  the  former  and 
not  enough  of  the  latter  during  the  period  of  my  career  when  1  worked  in 
the  intelligence  arena.  The  second  certainly  provides  clarification  and 
ease  of  use... obviously;  if  1  were  still  in  the  field  1  would  prefer  the 
second  product. 

60 

It  would  be  more  useful  if  he  links  were  established  as  credible  reliable 
sources.  We  get  lulled  into  using  hyperlinks  not  checking  the  veracity  of 
the  poster  for  the  hyperlink...  assuming  the  author  did  that  vetting  for 
us.  Being  intelligence  professional  skepticism  runs  deep  so  1  would 
want  to  verify  not  only  the  initial  report  but  any  additional  information 
referenced.  1  would  caution  hyperlinks  are  not  a  panacea  unless 
standards  are  established  to  ensure  the  entire  product  (including  the 
links)  are  verified  and  trusted  as  being  accurate. 

61 

Although  intel  doesn't  always  have  judgments  or  conclusions,  it  is 
certainly  more  useful  that  anecdotal  information.  1  thought  the 
hyperlinks  help  to  develop  a  better  understanding  of  the  information 
that  was  presented  in  the  sample. 

62 

Until  1  participated  in  your  survey,  the  Animal  Rights  activists  were  just 
marginally  part  of  my  perception.  You  changed  my  mind  with  the 
hyperlinks  since  1  learned  that  some  of  these  groups  target  the  nuclear 
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sector.  Very  useful  for  a  manager  who  likes  to  get  relevant  info  quickly, 
with  enough  depth  to  be  certain  that  the  information  is  credible,  but 
without  a  lot  of  keystrokes. 

63 

Hyperlinked  product  is  excellent  and  easy  to  use. 

64 

More  information  is  available  through  the  hyperlinks,  ease  of  access, 
ability  to  quickly  gather  historical  data  etc. 

65 

Good-luck  with  your  research.  1  feel  strongly  that  using  hyperlinks  will 
enhance  not  only  the  effectiveness  but  will  also  limit  "unnecessary" 
time  researching  documents,  i.e.,:  statutes,  agreements,  codes  etc. 

66 

The  system  is  as  good  as  the  intel 

67 

Hyper-links  are  much  more  beneficial  to  time  saving  and  a  quick  up  to 
snuff. 

68 

One  of  the  hyperlinks  did  not  have  the  correct  information  when  clicked 
on.  1  believe  it  was  "Kevin  Kjonaas."  Putting  too  many  hyperlinks  could 
cause  distraction  however,  so  1  believe  there  is  a  tradeoff  between 
providing  too  much  information  and  timely  information. 

69 

If  this  survey  results  in  hyper-links  being  included  in  future  intelligence 
briefings,  this  will  have  been  worth  the  time  spent  on  the  survey  and 
would  greatly  enhance  local  gov't's  ability  to  plan  and  respond  more 
effectively. 

70 

Hyperlinks  will  definitely  improve  knowledge  in  areas  that  small 
agencies  don't  normally  have  the  time  to  spend  on,  and  add  credibility 
to  the  information  being  reported. 

71 

None 

72 

It  was  like  night  and  day! 

73 

The  type  of  hyperlink  is  more  important  than  just  having  the  links 
available.  Some  hyperlinks  are  not  valuable  or  add  value  to  the  info 
block.  Hyperlinks  that  help  with  additional  insight,  pros/cons,  or  can 
help  link  one  event  with  another  is  most  helpful. 

74 

Hyperlinks  are  the  way  to  go.  The  take  full  advantage  of  today's 
technology. 

75 

Good  concept,  people  will  seek  out  more  information  when  it  is  easily 
accessible.  Perhaps  the  difference  between  a  more  though  out, 
researched  product  versus  a  mediocre  survey  of  information. 

76 


As  a  policy  maker  I  don't  see  much  intelligence;  however  I  do  see  the 
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benefit  of  having  intelligence  with  additional  background  information  for 
a  non-intel/law  enforcement  policy  maker  can  grasp  a  better  picture  of 
the  situation. 

77 

Nice  concept  that  allows  for  greater  collaboration 

78 

Would  hyperlinked  data  go  through  the  Internet?  If  so,  how  is  the 
transmission  of  classified  data  to  be  secured? 

79 

The  use  of  hyperlinked  intelligence  (AND  OTHER  REPORTS)  is  a 
great  idea.  Hopefully,  this  will  be  the  way  of  the  future. 

80 

It  is  clearly  better  with  hyperlinks.  It  cuts  down  on  investigative 
background  time. 

81 

PHOTOS  AND  PRIMARY  UNIT  OR  POINT  OF  CONTACT  IS  ALWAYS 
NEEDED  HOWEVER  LACKING  IN  QUITE  A  FEW  INTEL  REPORTS. 

AS  MUCH  AS  POSSIBLE  ONE  LEAD  UNIT  OR  AGENCY  SHOULD 

BE  IDENTIFIED  AS  MORE  AND  MORE  WE  ARE  STEPPING  ON 

EACH  OTHERS  TOES  AND  OVERLAPPING. 

82 

Great  idea! 

83 

Second  sample  give  more  knowledge. 

84 

Hyperlinks  to  large  files  take  some  time  to  download,  which  breaks  the 
continuity  of  information  flow.  If  the  hyperlinks  were  to  edited 
summaries  of  Intel  info,  it  would  facilitate  more  timely  downloads  and 
reading. 

85 

1.  Being  in  Emergency  Management,  intelligence  are  not  a  regular 
feature  of  my  job  -  but  they  should  be. 

2.  Hyperlinks  are  only  good  if  current,  and  as  a  report  ages,  links  are 
likely  to  become  inactive.  Also,  links  to  a  specific  item  within  a  web 
page  are  more  efficient  than  a  generic  link  to  a  web  page  -  in  that  you 
then  have  to  spend  time  searching  for  the  info  you  want. 

86 

Intelligence  analysts  are  required  to  sift  through  voluminous  amounts  of 
reports  and  data  to  develop  an  assessment.  Any  tools  —  such  as  the 
use  of  hyperlinks  ~  that  can  be  incorporated  to  facilitate  speed  of 
analysis  should  be  utilized. 

87 

As  an  emergency  manager  my  needs  for  intelligence  generally  center 
around  knowing  the  where's  and  when's  of  a  potential  incident  —  the 
sources,  methods,  leads,  and  general  law  enforcement  and  intel 
background  information  is  of  little  value  to  me. 

88 

At  first  1  was  not  sure  just  what  you  were  asking.  1  may  not  be  on  a  front 
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line  organization,  however,  I  would  find  this  information  important  and 
needed. 


89 

The  hyperlinked  product  is  superior.  It  effectively  provides  the  means  to 
answer  questions  thus  eliminating  subsequent  searching. 

90 

none 

91 

The  product  didn't  contain  recommendations  or  conclusions  so  it  was 
difficult  to  rate  the  product  on  those  criteria. 

92 

Great  idea,  and  while  hyperlinking  will  require  more  front-loading  of 
information,  will  significantly  increase  the  usefulness  of  the  product  and 
will  significantly  reduce  additional  research  required  by  the  analyst. 

93 

Your  agree. ...and  disagree  were  at  the  wrong  ends....1  should 
represent  Agree. 

94 

none 

95 

None 

96 

Good  Luck!!! 

97 

Very  illustrative  survey  -  well  done! 

98 

Hyperlinks  allow  for  shorter  executive  summaries  as  well  as  more 
details  if  needed.  It  also  allows  for  different  levels  of  security.  I.e.,  Exe 
Sum  at  the  FOUO  and  Hyperlinks  at  a  more  secure  level. 

99 

Great  concept.  Look  forward  to  reading  the  thesis  when  it  is  published. 

100 

Honestly,  I'd  consider  all  my  responses  to  be  outliers  and  should  not  be 
considered  in  your  thesis.  Reasoning:  1  don't  use  intelligence  products 
in  my  day  to  day  activities.  We  create  emerging  technology  solutions  to 
support  homeland  security.  The  entities  1  support  would  most  likely  be 
valuable  in  guiding  and  developing  algorithms  that  could  be  utilized  to 
automatically  scan  and  mine  large  volumes  of  material  to  help  develop 
the  intelligence  product  that  you  would  communicate  to  users.  1  would 
have  considered  prescreening  questions  at  the  beginning  of  the  survey 
to  include/exclude  survey  responders  based  upon  their 
need/use/interaction  with  intelligence  products.  The  survey  appears 
that  it  would  be  very  useful  for  those  it  pertains  to.  Best  of  luck! 

101 

None 

102 

Be  careful  of  hyperlinking  too  much. ..sometimes  that  is  a  distracter  and 
may  drive  the  reader  down  the  wrong  "rabbit  hole"  as  they  start 
following  links  away  from  the  main  story/product 
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The  "hyperlinks"  were  great!  The  initial  direction  (first  paragraph)  of  the 
scenario  was  so  vague,  however,  that  it  made  it  difficult  to  comment  on 
effectiveness.  From  a  Statewide  perspective  and  having  566 

103  municipalities  the  intelligence  dealing  with  a  single  meeting  within  the 
next  2  months  anywhere  in  the  State  is  hard  to  follow-up  on.  If  the  point 
of  the  survey  was  to  test  intelligence  with  and  without  the  hyperlink  -  it 
is  clear  that  the  hyperlink  product  is  very  worthwhile  and  informative. 

I  am  a  health  physicist  and  basically  only  care  about  radiation  -  the 
hyperlinks  would  allow  me  to  rapidly  determine  if  there  was  a  history  or 
"connection"  to  radiation.  If  a  suspect  had  a  degree  in  physics  and  we 

104  had  a  report  of  stolen  radioactive  material  -  we  might  start  pulling  the 
tread.  The  hyperlinks  would  give  any  body  a  better  chance  of  seeing  a 
connection  to  their  field.  Downside  -  it  could  take  a  lot  of  time  to  surf 
through  all  the  info. 

Hyperlinks  are  a  valuable  tool  in  being  able  to  reinforce  conclusions  or 
facts;  however,  it  is  important  that  the  links  be  to  credible,  known 
sources.  I  would  take  some  stock  into  a  source  that  was  linked  to  in  a 

,|05  government  briefing  more  than  I  would  by  searching  Google  for 
information  on  a  subject.  However,  it  would  be  important  that  the 
briefings  only  contain  credible  sources  that  the  governmental  agency 
issuing  the  briefing  has  validated  the  information  that  the  sources 
contain. 

lOg  Superb  use  of  inteL.the  more  information  provided,  the  greater 
opportunity  for  decisions  to  be  made  that  affect  local  operations. 

107  Good  tool  for  an  alert  with  the  hyperlinks  allowing  more  time  for 
research  into  possible  threats  in  local  area 

Hyperlinks  are  great  for  providing  background  information  -  but  it  is  not 

10g  hyperlinks  that  are  the  problem  -  the  problem  is  the  quality  of 

intelligence  and  the  availability  of  actionable  information.  For  example, 
in  this  example  the  information  is  not  actionable  for  my  organization. 

109  Hyperlinks  provided  quick,  useful  information  when  needed. 
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Naval  Postgraduate  School 

Center  for  Homeland  Defense  and  Security 

Intelligence  Research 


Questions  marked  with  an  asterisk  (*)  are  required. 
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Center  for  Homeland,Defense  and  Security 

THE  NATION'S  HOMELAND  SECURITY  EDUCATOR 
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|.fcl  Done 


Please  read  this  condensed  sample  unclassified  Intelligence  Product.  Seven  questions  on  the  quality  and  value  of  this  product  will  be 
asked  after  you  have  completed  reading  this  sample.  You  may  come  back  to  this  page  via  the  back  button  at  any  time. 

SAMPLE  -  FOR  RESEARCH  ONLY 

The  State  Police  have  received  information  that  environmental  terrorists  may  have  scheduled  a  long  term  strategic  planning  session  in 
the  state  within  the  next  60  days.  Please  contact  your  local  law  enforcement  or  the  FBI  Joint  Terrorism  Task  Force  with  any 
information  you  may  receive  concerning  this  meeting  or  contacts  with  known  or  suspected  environmental  terrorist  group  members. 

In  March,  a  closely  watched  trial  in  New  Jersey  ended  with  the  convictions  of  Stop  Huntingdon  Animal  Cruelty  (SHAC)and  six  of  its 
members  for  using  the  group's  Web  site  to  incite  violence  against  people  associated  with  Huntingdon  Life  Sciences,  a  firm  that  tests 
drugs  on  animals.  Victims,  who  had  personal  information  about  them  and  their  families  posted  on  the  Web  site,  testified  that  they  had 
received  death  threats,  seen  their  windows  smashed  and  cars  overturned,  and  had  their  children  threatened,  among  other  things.  The 
six,  who  now  face  up  to  seven  years  in  prison,  were  the  first  ever  charged  under  the  Animal  Enterprise  Protection  Act,  a  law  that  was 
amended  in  2002  to  include  "animal  enterprise  terrorism." 

One  of  those  convicted  was  Kevin  Kjonaas,  who  was  president  of  SHAC  until  his  indictment  in  the  case  in  2004.  Jurors  heard 
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evidence  that  Kjonaas  had  made  a  call  to  Daniel  Andreas  San  Diego  just  hours  after  San  Diego  allegedly  bombed  two  California 
businesses  in  August  2003,  although  Kjonaas  was  not  charged  with  any  crimes  in  connection  with  the  bombings.  The  FBI  has  offered 
a  $250,000  reward  for  the  arrest  of  San  Diego,  who  has  tattoos  of  burning  buildings  on  his  chest. 

On  Jan.  19,  another  major  case  was  brought  against  1 1  people  accused  of  terrorist  attacks  in  Oregon,  Wyoming,  Washington, 
California  and  Colorado  on  behalf  of  the  Animal  Liberation  Front  (ALF)  and  the  Earth  Liberation  Front  (ELF)  In  a  65-count 
indictment  returned  by  a  grand  jury  in  Oregon,  the  11,  who  called  themselves  "The  Family,"  are  accused  of  17  attacks,  including 
sabotaging  a  high-tension  wire  and  destroying  a  Vail,  Colo.,  ski  resort,  in  a  $12  million  arson. 


3  *The  Product  was  timely  and  relevant  to  your  mission,  programs, 
priorities,  or  initiatives. 


Strongly 

Disagree 

Strongly  Agree 

N/A 
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4  *The  product  was  clear  and  logical  in  the  presentation  of  information  with 
supported  judgements  and  conclusions. 
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*The  product  is  reliable;  i.e.  sources  well  documented  and  reputable. 
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Questions  marked  with  an  asterisk  (*)  are  required. 
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Naval 

postgraduate 

School 

Center  for  HomelandBefense  and  Security 

tiie  nation's  homeland  Security  educator 

Please  read  this  condensed  sample  unclassified  Intelligence  Product.  Nineteen  questions  on  the  quality  and  value  of  this  product  will 
be  asked  after  you  have  completed  reading  this  sample.  You  may  come  back  to  this  page  via  the  back  button  at  any  time. 


SAMPLE  -  FOR  RESEARCH  ONLY 

The  State  Police  have  received  information  that  environmental  terrorists  may  have  scheduled  a  long  term  strategic  planning  session  in 
the  state  within  the  next  60  days.  Please  contact  your  local  law  enforcement  or  the  FBI  Joint  Terrorism  Task  Force  with  any 
information  you  may  receive  concerning  this  meeting  or  contacts  with  known  or  suspected  environmental  terrorist  group  members. 

In  March,  a  closely  watched  trial  in  New  Jersey  ended  with  the  convictions  of  Stop  Huntingdon  Animal  Cruelty  (SHAC)  (page  80  of 
280)  and  six  of  its  members  for  using  the  group's  "Web  site  to  incite  violence  against  people  associated  with  Huntingdon  Life 
Sciences  ,  a  firm  that  tests  drugs  on  animals.  Victims,  who  had  personal  information  about  them  and  their  families  posted  on  the  Web 
site,  testified  that  they  had  received  death  threats,  seen  their  windows  smashed  and  cars  overturned,  and  had  their  children 
threatened,  among  other  things.  The  six,  who  now  face  up  to  seven  years  in  prison,  were  the  first  ever  charged  under  the  Animal 
Enterprise  Protection  Act ,  a  law  that  was  amended  in  2002  to  include  "animal  enterprise  terrorism." 


One  of  those  convicted  was  Kevin  Kjonaas  ,  who  was  president  of  SHAC  until  his  indictment  in  the  case  in  2004.  Jurors  heard 
evidence  that  Kjonaas  had  made  a  call  to  Daniel  Andreas  San  Diego  just  hours  after  San  Diego  allegedly  bombed  two  California 

businesses  in  August  2003,  although  Kjonaas  was  not  charged  with  any  crimes  in  connection  with  the  bombings.  The  FBI  has  offered zJ 
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One  of  those  convicted  was  Kevin  Kjonaas  ,  who  was  president  of  SHAC  until  his  indictment  in  the  case  in  2004.  Jurors  heard 
evidence  that  Kjonaas  had  made  a  call  to  Daniel  Andreas  San  Diego  just  hours  after  San  Diego  allegedly  bombed  two  California 
businesses  in  August  2003,  although  Kjonaas  was  not  charged  with  any  crimes  in  connection  with  the  bombings.  The  FBI  has  offered 
a  $250.000  reward  for  the  arrest  of  San  Diego,  who  has  tattoos  of  burning  buildings  on  his  chest. 

On  Jan.  19,  another  major  case  was  brought  against  1 1  people  accused  of  terrorist  attacks  in  Oregon,  Wyoming,  Washington, 
California  and  Colorado  on  behalf  of  the  Animal  Liberation  Front  (ALF)  and  the  Earth  Liberation  Front  (ELF)  In  a  65-count 
indictment  returned  by  a  grand  jury  in  Oregon,  the  11,  who  called  themselves  "The  Family,"  are  accused  of  17  attacks,  including 
sabotaging  a  high-tension  wire  and  destroying  a  Vail,  Colo.,  ski  resort,  in  a  $12  million  arson  (page  18  of  68j 
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13  The  product  would  contribute  to  satisfying  intelligence  gaps  or 
predicating  cases  or  intelligence  operations,  especially  previously 
unknown  areas. 
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14  The  product  would  result  in  a  change  in  investigative  or  intelligence 
priorities  and/or  a  shift  from  unaddressed  to  addressed  work,  or  vice 
versa. 
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21  *Using  intelligence  products  with  hyperlinks  would  make  it  easier  to  do 
my  job. 
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making  information. 
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